Hotspot & Celebrity Problem Lab (Interactive)
Publish as an 80M-follower celebrity, salt a 100k/sec like counter, and toggle L1 micro-caches to defuse partition skew. Model Zipfian traffic skew: push versus pull versus hybrid fan-out, salted distributed counters, replicated hot keys, and 2-second in-process caches.
Celebrity Hotspot & Salted Counter Lab
Zipfian skew breaks hash partitioning — tune fan-out mode, salting, and L1 micro-cache to defend it.
How It Works Under the Hood
Hash partitioning assumes uniform access, but real traffic is Zipfian: the top 0.01% of keys take over half the load, pinning one shard at 100% CPU while 99 idle. A pure push fan-out for an 80M-follower post means 80M LPUSH operations — 1,600 seconds of queue backlog. Production defenses are hybrid fan-out (push normal users, merge celebrity posts on read in under 2 ms), write-side key salting across N sub-counters with scatter-gather reads, read-side hot-key replication, and a 2-5 second L1 micro-cache that absorbs stampedes locally.
Core Architectural Principles
- Hybrid fan-out classifies users at a ~25k follower threshold into push versus pull-then-RAM-merge paths.
- Salting a viral counter across 16 buckets divides single-key write load; reads sum buckets with MGET.
- Adding nodes does not fix a hotspot — the same key still hashes to the same shard regardless of cluster size.
When designing any feed, raise hotspots unprompted: "Normal users fan out on write; celebrities above 25k followers are merged at read time by a Timeline Mixer." For counters, specify salted sub-keys with scatter-gather aggregation, and mention a two-second Caffeine micro-cache for breaking-news reads. That trio signals production experience over textbook recall.
Hybrid fan-out and salting protect clusters from write amplification and lock contention but add dual code paths and slightly slower scatter-gather reads that uniform-traffic B2B systems never need.