Split-Brain & Fencing Mechanisms
Prevent catastrophic dual-master partition states: Majority quorums, STONITH (Shoot The Other Node In The Head), and generation epochs.
01.The Split-Brain Disaster in Distributed Systems
In distributed architectures with primary-follower or active-passive topologies, high availability depends on automatically promoting a standby node to primary whenever the active primary crashes.
However, consider what happens when a network partition severs communication between two datacenters (e.g., Datacenter 1 and Datacenter 2) while all individual machines remain healthy:
- Datacenter 1 (Old Master): Believes Datacenter 2 is down and continues accepting client writes.
- Datacenter 2 (Standby): Heartbeats from Datacenter 1 time out; Datacenter 2 assumes the primary has crashed and promotes its local node to Primary.
- The Split-Brain State: Both nodes now believe they are the authoritative Primary. Both accept conflicting
INSERT,UPDATE, andDELETEtransactions from clients.
When the network partition heals, the two database datasets have diverged irreparably with conflicting primary keys and modified balances, resulting in irrecoverable data corruption.
Split-Brain Anomaly & Quorum Prevention 🧠
Split-Brain Anomaly & Quorum Prevention 🧠
Network partition isolates Node A from Nodes B & C.
Unlock Topic #91: Split-Brain & Fencing Mechanisms
You are viewing a preview. The full in-depth engineering deep dive, interactive simulators, architecture flowcharts for this topic, along with self-assessment quizzes, are available with Pro or Lifetime Access.
Failure modes, high-throughput bottlenecks, and real FAANG implementation decisions.
Interactive system topology diagrams, live parameter simulators, and downloadable SVG charts.
Staff-level multiple-choice quiz questions with instant feedback and answer explanations.
Firebase Google authentication automatically syncs your completed topics and quiz scores.
How clear and actionable was this distributed systems breakdown?