Distributed Locks: Redlock & ZooKeeper

In distributed architectures, multiple worker services frequently need mutual exclusion on shared resources—such as ensuring a single cron job runner processes payroll, or preventing two concurrent checkout threads from double-spending a customer's gift card balance.

A common pattern is acquiring a distributed lock in Redis with a Time-To-Live (TTL):

bash
Copy
SET lock:resource_123 "client_A_uuid" NX PX 10000

The Stop-the-World GC Hazard:

As distributed systems expert Martin Kleppmann famously demonstrated, TTL-based locks without fencing tokens are inherently unsafe:

1. Client 1 acquires the lock with a 10-second TTL.
2. Client 1 immediately hits an unexpected 15-second Stop-the-World JVM Garbage Collection pause, hypervisor CPU freeze, or disk paging stall.
3. While Client 1 is frozen, the 10-second TTL expires on Redis, and Redis deletes the lock key.
4. Client 2 requests the lock, successfully acquires it, writes an update to the database, and commits.
5. Client 1 wakes up from its GC pause. Believing it still holds the lock (since it executed no release logic), it blindly executes its database write, silently corrupting or overwriting Client 2's data!