Chat Gateway Sharding Lab (Interactive)
Scale online sockets across gateway shards and size memory and offline push load. Convert DAU and online share into long-lived socket connections per gateway, message storage bytes, and offline push fallback.
WhatsApp-scale Chat: Socket Fleet, Registry & Fan-Out
Size the stateless Gateway tier that holds WebSockets, route via the Redis session registry, and pay for group fan-out.
› Client → LB → Gateway G-128 holds Alice's socket. Registry lookup: session:{bob} → G-77.
› G-77 pushes over its socket; ACK flips status sent→delivered→read across services.
Bob offline (9,600M such DMs/day): persist to Cassandra, fire APNs/FCM wake-up, deliver on reconnect.
› Group msg: fan-out ×8 recipients → 96B total deliveries/day; history partitioned by chat_id, clustered by time-uuid message_id.
Gateways are deliberately stateless about message data — they only own sockets. The Redis Session Registry is the indirection that lets any Chat Routing Service accept a message for a user it isn't directly connected to. Wide-column stores absorb the append-heavy chat history, and per-recipient delivery state (sent/delivered/read) is tracked with message IDs, never assumed from a 200 OK.
How It Works Under the Hood
A chat system's hard constraint is connection memory, not compute. Online users hold open WebSocket connections at roughly 10 KB each, and a single gateway tops out around 500k sockets, so millions of concurrent users force a shard count and a routing layer that maps user to gateway via presence. Throughput flows through the message store — about 200 bytes per message in Cassandra partitioned by conversation makes daily storage a function of messages sent and group share, since a 1-to-N group fans one write into N inboxes. Offline recipients cannot use the socket, so their messages must fall back to a push gateway. Sweep online share and message rate to watch sockets, storage, and push load move.
Core Architectural Principles
- Sockets equal online users; gateways equal ceil(sockets / 500k) with about 10 KB RAM per live connection.
- Message storage = msgs/sec x 200 B x 86,400, and group messages fan into per-member inboxes.
- Offline recipients bypass the socket tier and route through APNs/FCM push fallback.
Separate the three concerns — long-lived connections in the gateway tier, message durability in a partitioned wide-column store keyed by conversation, and presence routing. Explain gateway sharding and reconnect with backoff, ordered delivery via sequence numbers, and the online/offline split. Quantify connection RAM and group-message write amplification to show scale awareness.
Stateful gateways give instant delivery but cost connection memory and complicate reconnect routing; stateless polling scales easier yet adds latency.