A Workflow Tracker for the EventBridge Pipeline¶

2026-09-12

In the EventBridge concept, each document has one service responsible for its next action. A workflow tracker turns that ownership into a view people can use.

Imagine opening the tracker and seeing these illustrative numbers:

Click Review to see its documents, oldest assignment first. Open invoice-42 to see “Waiting for approval, assigned at 09:20” and the handoffs that brought it there.

“Assigned” includes queued, running, and waiting work. It measures business responsibility, not worker concurrency or SQS queue depth. A review waiting on a person still belongs to review.

The schema below is one possible way to support this view with DynamoDB. The document pipeline remains an illustrative example.

Where the tracker gets its information¶

The shared emit API verifies the sender against the current owner, then records nextOwner as the new owner when accepting a handoff. The tracker can read that record through a query API:

flowchart LR
    Service -->|handoff| Emit[emit API]
    Emit -->|record accepted state| DB[(DynamoDB)]
    DB -->|outbox publisher| Bus[EventBridge]
    Bus --> Queue[SQS]
    Queue --> Next[Next service]
    UI[Workflow tracker] --> API[Tracker query API]
    API -->|read state and counts| DB

The tracker needs no calls to each processing service to assemble the current stage. Services report progress through the shared contract; the tracker displays the last accepted state. If the UI needs to distinguish queued from running work, the service must report when it starts.

Record the state change and an outgoing event intent together, then publish the intent to EventBridge. This transactional outbox pattern covers a crash between saving the handoff and publishing its event. The recorded owner may therefore have work whose delivery is still pending.

A separate tracker could instead subscribe through EventBridge and SQS and build its own view. That adds projection delay and requires handling duplicate and out-of-order events. Reading the existing accepted state is the simpler starting point for this concept.

Implementing emit¶

The emit contract needs to accept or reject a handoff reliably. Several implementations can provide that contract:

An HTTP implementation could return structured errors with status codes such as 409 Conflict for an ownership or version mismatch. API Gateway's Lambda proxy integration lets the handler specify the HTTP response.

For this concept, a small client wrapper around synchronous Lambda invocation is a useful starting point. The service waits for the acceptance result before treating its handoff as complete:

Service calls emit(...)
  → synchronously invoke handoff Lambda
  → check for an existing receipt for this request
  → validate sender and requested transition
  → transaction: check owner/version, update state,
                 write history, counters, receipt, and outbox
  ← acceptance receipt or rejection

Outbox publisher → EventBridge → next service

Owner and version checks remain conditions inside the transaction, so concurrent calls cannot both advance the same state version. A successful response means the handoff is durably accepted. EventBridge publication and the next service's work happen afterward.

Turning rejection into a caller exception¶

Lambda's synchronous RequestResponse invocation returns the function's result. A function error can arrive with HTTP status 200 and a FunctionError indicator, so the client wrapper must inspect the response rather than relying only on SDK exceptions. See Lambda Invoke.

Expected validation failures can use structured results that emit translates into local exceptions:

Handoff Lambda returns:  rejected, code=OwnershipMismatch
Client emit() raises:    OwnershipMismatch

Handoff Lambda returns:  accepted, version=8, receipt=...
Client emit() returns:   acceptance receipt

The wrapper also handles FunctionError responses and invocation failures. This gives callers a normal function-call interface while keeping the shared acceptance logic in one place.

A known ownership or version rejection means this handoff did not change state. A timeout or broken connection leaves the outcome uncertain: the transaction may already have committed. Retry the identical request with the same application event ID to recover its original receipt. Do not generate a new event ID for that retry.

Binding sender to the caller¶

The sender field expresses which service claims to be handing off. Permission to invoke a shared Lambda does not by itself prove that this field names the caller correctly.

The implementation needs a trusted binding between caller identity and logical service name. For example, an authenticated HTTP entry point can map its verified principal to a service identity. A direct-invocation design could use a separate IAM-restricted entry function for each service; that function supplies the fixed sender identity to the shared acceptance handler. Restrict access to the shared handler accordingly.

Whichever entry point is used, establish the sender identity before comparing it with the stored owner. The owner/version check then determines whether that service is allowed to advance this particular document.

Start with the questions¶

A stage is represented by owner; status describes progress within that stage. For example, review could have waiting-for-approval or decision-recorded. A completed document has a terminal status and no active owner.

A possible DynamoDB table¶

Use a table called Workflow with string keys PK and SK. Keep one current-state item per document, history beside it, and separate count items:

The current-state item might look like this:

PK: DOC#invoice-42
SK: STATE
owner: review
status: waiting-for-approval
version: 7
assignedAt: '2026-09-12T09:20:00Z'
dueAt: '2026-09-13T09:20:00Z'

version increases for each accepted state change. History uses that version, padded to a fixed width, to preserve the accepted sequence. assignedAt changes when a new assignment begins; an ordinary progress update keeps it unchanged so the tracker does not reset the waiting age.

For clarity, these examples assume one workflow run per document in one scope. Multiple tenants need tenant-scoped keys and authorization. If reprocessing is supported, include run identity in state, history, and handoffs, and decide whether counts represent active runs or distinct documents.

Transition versions and replay¶

The handoff API assigns a monotonically increasing version for each accepted state change within a document's workflow run. Store it in the current state, history, and outgoing event in the same transaction. The initial assignment gets version 1; the next accepted transition gets version 2. A rejected request or a retry of an accepted request creates no new version.

For example, review receives an assignment at version 7 and completes it with:

emit("DocumentApproved", documentId="invoice-42",
     sender="review", nextOwner="finalization", expectedVersion=7)

The API checks the owner and expected version, accepts the transition as version 8, and includes version 8 in the event sent to finalization. If a service reports intermediate progress, that also advances the state version; its later completion must use the version returned by that accepted update.

Keep a stable application eventId in the event payload alongside the version. Check for an existing acceptance receipt first: retrying the same request returns the original result even if ownership has since moved. A reused ID with different contents is an error. The version condition protects against a different, stale request.

What happens during an EventBridge replay?¶

Suppose DocumentApproved at version 8 has already been handled, and an archive replay delivers it again. The event retains its application event ID, document/run identity, and version. It is another delivery of the original transition, so no new transition version is allocated just for replay.

EventBridge can replay an archive to selected rules, but events are not guaranteed to replay in their original order. The application therefore needs checks appropriate to each consumer:

A “highest version seen” check is suitable for a latest-state projection only when each event carries the complete state that projection needs. Delta-based projections must handle gaps and ordering, or rebuild from accepted history. A subscriber matching only some event types will naturally see gaps in the document sequence.

For workers, recording an event as seen is not enough to prove its action finished. Scope completion records by consumer and application event ID, and coordinate concurrent attempts with a conditional claim and a recovery path for abandoned work. When an external action cannot share the completion transaction, use a stable downstream idempotency key or reconcile the outcome before repeating it. Owner/version checks prevent stale handoffs; they do not undo a duplicated external action.

Keep receipts and completion records for the supported replay period. Otherwise, an old event can look new after its deduplication record expires. Likewise, the document's current version alone cannot prove an independent subscriber has processed an older event.

Rebuilding a tracker versus rerunning the workflow¶

To rebuild a separate tracker projection, replay to its rule with fresh projection state and leave business-action consumers out of that replay. Full snapshots let the projection converge on the highest version even when delivery is out of order. Keep the authoritative workflow state and acceptance receipts intact.

To deliberately process a document again, create a new workflow run. Its versions may start at 1 because comparisons and deduplication are scoped to the run. Replaying an existing run preserves the identity of its original work.

Indexes for service views¶

A global secondary index (GSI) provides another way to query the same items. Add these derived attributes to active current-state items:

Use a consistent UTC timestamp format. The document suffix breaks ties. Only populate deadline keys when there is a deadline; remove both sets of index keys when a document finishes. History and count items omit these attributes, so they do not enter the indexes. This uses DynamoDB's sparse index behavior.

Project the document identifier, owner, status, assigned time, and deadline into the indexes so a list can display them without fetching every document separately.

Conceptually, the tracker reads:

Document details:  GetItem(DOC#invoice-42, STATE)
Review work:       Query ByOwner where ownerPK = OWNER#review
Overdue reviews:   Query ByOwnerDue where duePK = OWNER#review
                   and dueSK < current UTC timestamp
Document history:  Query PK = DOC#invoice-42, SK begins with EVENT#

The owner query returns oldest assignments first. Paginate lists as they grow. The deadline query above treats a deadline strictly before now as overdue. For all services, run that query for each known owner.

GSI reads are eventually consistent: a document can briefly appear under its previous owner after a handoff. The detail view can use a strongly consistent base-table read. Before acting on a listed document, validate its current state and version. See DynamoDB index consistency.

Counts without reading every document¶

For a small prototype, the tracker could count the owner query's results. However, Select=COUNT still consumes read capacity and requires pagination beyond the query page limit; it is not a constant-cost aggregate. A changing, paginated index also does not provide a single snapshot. See DynamoDB query counts.

For a frequently refreshed dashboard, maintain a count per owner. When a document moves from routing to review, accept these changes in one transaction:

Verify the authenticated caller matches sender.
Require the stored owner to equal sender and the version to match.
Set its owner to review and increment its version.
Decrease routing's count by one.
Increase review's count by one.
Append history and record the outgoing event intent.
Record a receipt for this handoff's stable event ID.

The owner and version checks are conditions on the state update inside the transaction. A wrong sender or stale version rejects the handoff without changing state, counts, or outgoing intents. Record the rejected attempt separately for troubleshooting. The version also prevents an old completion from being accepted if the document later returns to the same service. The initial upload instead requires permission to create the workflow and a condition that its state does not already exist.

Initial assignment increments only the first owner's count; completion decrements only the last owner's count. Progress within the same assignment leaves counts unchanged. Every ownership-changing path must use the same accounting.

A repeated handoff returns its existing receipt and does not change counts again. The receipt must be conditionally created in the same transaction; a preliminary lookup alone would not prevent concurrent duplicates. These are application-maintained transactional counts, following the approach described in AWS's item-count guidance.

The tracker reads the small set of count items for its overview. A transactional read can give a consistent snapshot across those items. The counts describe accepted assignments; the GSI lists may briefly lag behind them.

Keeping the starting point manageable¶

Start with document lookup, an owner list, and assignment age. Add the deadline index when the tracker needs overdue views, and stored counters when repeatedly counting lists becomes expensive.

A busy owner can concentrate index writes and counter contention. At higher volume, partition by tenant or workload, then consider sharded owner indexes and counters; the tracker must combine the resulting reads. Another option is asynchronous dashboard counts, with a visible refresh time and reconciliation process.

The tracker makes service responsibility visible. A document assigned to review for two days gives an operator a concrete place to investigate, while service counts show where work is accumulating.