Message Ordering & Partition Key Lab (Interactive)
Choose a partition key, send events, and watch ordering, skew, and retry inversions bite. Batch-send keyed events through hash(key) modulo partitions to see per-key FIFO ordering, hot-partition skew, key salting effects, and retries breaking global order.
Partition Keys, Order & Hot-Partition Skew
Route bank events by MurmurHash(key) % P, crank a flash-sale hot key, and let retries invert sequence — or fix it with idempotence.
Events Routed
0
across 3 partition(s)
Hottest Partition
0%
balanced target 33%
Skew (CV)
0.00
>0.5 = one consumer melting
Order Inversions
0
withdraw before deposit!
idle while neighbors burn
idle while neighbors burn
idle while neighbors burn
Per-key order costs nothing to scale: unrelated accounts run in parallel, and hash collisions between accounts are harmless. The two real hazards are above: a hot key (raise it to 16/20 and salt it away) and retry inversion (turn idempotence on).
How It Works Under the Hood
Brokers only guarantee ordering where they serialize: Kafka commits records per partition in append order, so picking the partition key picks the ordering scope. Hash(order_id) keeps one order’s events causal but lets different orders interleave—usually exactly what you want. A skewed key (one mega-merchant) melts a single partition, fixed by salting the key into sub-partitions, which then sacrifices even the per-key guarantee across salts. Retries breaking ordering are the sneaky failure: an in-flight resend overtakes its own successor.
Core Architectural Principles
- MurmurHash(key) % partitions routes every record; same key always lands on the same ordered log.
- Order violations appear only across partitions—or within a key once an in-flight retry overtakes the next event.
- Salting keys into N buckets flattens hot-partition load but scatters one key’s ordering across buckets.
Answer "global ordering at scale is a myth—design the key so per-entity ordering is all you need." Name the hash(key) % partitions mechanic, warn that adding partitions reshuffles the mapping, and list fixes for skew: better keys, salting with a reassembly step, or enable.idempotence to kill retry-induced inversions.
Fine-grained keys maximize parallelism and minimize skew but only order within each key; coarse keys order more but serialize throughput.