Bottlenecks & Trade-Offs Lab (Interactive)
Mitigate six distributed failure vectors and assemble the trade-off sentence to earn Staff-level calibration. Pick a mitigation for SPOFs, connection exhaustion, replication lag, hotspots, cache stampedes, and cascading failures, then complete the four-part trade-off formula; the readout maps your coverage to L4, L5, or L6+ signals.
6-Bottleneck FMEA Audit
Audit your baseline architecture: pick a mitigation for each failure vector and assemble the 3-part trade-off sentence to earn seniority signal.
1. Single Point of Failure
One DB primary / gateway / broker leader; its crash is a total outage.
2. Connection & IOPS Exhaustion
500 app instances x 20 sockets = 10,000 TCP connections saturating PostgreSQL memory and WAL disk IOPS.
3. Replication Lag / Stale Reads
User writes to primary, reloads from a replica 300 ms behind, thinks their update was lost.
4. Hotspot Keys & Celebrity Skew
A 50M-follower account: push fan-out melts Kafka/Redis; pull reads melt one shard.
5. Cache Stampede / Thundering Herd
A hot key expires under 20,000 QPS; every concurrent request misses at once and brown-outs the DB.
6. Cascading Failure
Downstream ML ranker slows 20 ms -> 3 s; gateway threads block, unrelated services starve.
Trade-off sentence builder
Each part earns +7 staff signal. "There are no solutions, only trade-offs."
- Mitigations correct
- 0 / 6
- Weak / avoided answers
- 0
- Unaudited failure vectors
- 6
- Sentence parts
- 0 / 4
Danger zone: design defended as "perfect"
Every unaudited vector is an objection the interviewer will raise for you. Name it first: "let me spend 3 minutes identifying the top bottlenecks in this design."
How It Works Under the Hood
The line between mid-level and Staff candidates is how they treat imperfection: L4 defends the design as flawless, L5 volunteers a systematic audit of where it breaks, and L6+ quantifies each trade-off unprompted. This lab drills that audit - six classic failure vectors, each with a correct production mitigation (PgBouncer pooling, LSN-based read-your-own-writes, hybrid push-pull fan-out, single-flight stampede locks, circuit breakers) and plausible weak answers. Residual unaudited vectors are previewed as interviewer objections, turning proactive self-critique into a visible numeric signal.
Core Architectural Principles
- Six failure vectors audited: SPOF, connection and IOPS exhaustion, replication lag, celebrity hotspots, thundering herd, cascading timeouts.
- Each vector offers one strong and two weak mitigations, scored like an interviewer grading coachability.
- The three-part trade-off sentence - choice, benefit, downside, mitigation - adds the Staff calibration bonus.
Right after the baseline architecture, say "let me spend three minutes auditing where this breaks." Always pair a bottleneck with its concrete mitigation - naming stampedes without single-flight reads as listing problems. Mention read-your-own-writes whenever async replicas appear, and frame every choice as chosen-A-over-B with an acknowledged downside; there are no solutions, only trade-offs.
Proactive auditing neutralizes objections and elevates hiring signals, but over-critiquing without mitigations undermines the design into incoherence.