Transactional Outbox Lab (Interactive)
Crash pods between INSERT and Kafka publish, then let Debezium and a dedupe table clean up at-least-once chaos. Execute 100-order batches under dual-write and outbox+CDC strategies, accumulating zombie orders, phantom events, and double charges until idempotent consumers filter duplicates.
Dual-Write vs Transactional Outbox
Place 100 orders while pods crash and brokers redeliver. Watch which architecture keeps DB state and Kafka events in lockstep.
Zombie orders
0
Phantom events
0
Double charges
0
Press Execute to run a batch of 100 orders.
0 orders · 0 delivered · 0 deduped
Coordination: choreography vs orchestration
Choreography event links
10
Orchestrator commands
10
Borderline. If any step needs a compensation chain, the event graph becomes spaghetti — consider a Temporal workflow.
How It Works Under the Hood
Publishing an event alongside a database write means two systems that share no transaction: whichever order you choose, a crash between the steps leaves a committed order whose event never shipped (a zombie nobody fulfills) or a published event whose order insert failed (a card charged for nothing). The transactional outbox folds the event into the local commit — orders row plus outbox row are atomic — and Debezium tails the write-ahead log to publish with under 10 ms lag. Because CDC delivers at-least-once, consumers still need a processed-events dedupe table to make replays idempotent.
Core Architectural Principles
- Batch runner draws per-order outcomes from a seeded PRNG: DB blips, publish-step crashes, and broker duplicates.
- DB-first and Kafka-first dual-writes produce zombies and phantoms respectively; the outbox produces neither.
- At-least-once redelivery hits the dedupe toggle: processed_events primary key absorbs duplicates safely.
When designing event-driven services, pre-empt the dual-write bug: "I never publish from application code after a commit — the event writes to an outbox in the same transaction and CDC streams it." Then own at-least-once: consumers are idempotent via a dedupe table or natural upsert keys. For multi-step workflows contrast choreography fan-out with orchestrated sagas and their auditability.
Outbox plus CDC guarantees state and events never diverge but adds table, lag, and consumer idempotency obligations.