Failure is part of the flow.
Trigger a failure, inspect the state transitions, and follow recovery to a known outcome. This is a local teaching model, not real distributed infrastructure.
Advance jumps to the next virtual event without waiting in real time. Replay uses the same idempotency key.
Make failure observable.
Create a transaction, then step through the scenario or run directly to its outcome.
Make recovery understandable.
Try again, with a budget.
A temporary failure can be retried. Exponential backoff begins at 200 ms and doubles between retries while the circuit is closed. A maximum-attempt budget prevents endless work. A permanent validation error stops immediately.
Stop calling an unhealthy service.
In the outage scenario, two consecutive failures open the circuit and queue the transaction. A modeled worker waits 2,000 ms before one half-open probe. Success closes the circuit; another failure can reopen it if attempts remain.
One key, one operation.
Replaying a request with the same key returns the existing state or stored outcome. It creates no second upstream attempt. In a real system this needs atomic persistence and concurrency controls; this model uses a single in-memory state machine.
A timeout can mean “unknown.”
If a response is lost after commit, retrying blindly may duplicate a side effect. The lost-reply scenario checks the modeled outcome store and marks the existing commit successful. This demonstrates a recovery pattern, not an exactly-once guarantee.
The notification project demonstrates an API/queue/worker boundary. Its case study explicitly identifies persistence, acknowledgement, and retry guarantees as further work.
Explore the notification system →