Page Cache & fsync Lab (Interactive)
Commit WAL transactions, then pull the plug — only fdatasync-ed writes survive the crash. Track dirty pages through write-back, per-commit fsync, and group-commit strategies, measuring ack latency, commit rate, and real RPO.
Page Cache, fsync & Durability Simulator
Commit WAL transactions, then pull the plug — only what reached non-volatile media survives.
Amortizes one 0.5 ms disk seek across 8 concurrent transactions.
Commit transactions to fill the dirty page cache.
write() returns once bytes land in the DRAM page cache (~100 ns); durability only begins at fdatasync(). That gap is why serial fsync-per-commit caps you near 2000 txn/s — group commit and Kafka's zero-copy sendfile() from the page cache are the standard escapes.
How It Works Under the Hood
write() returns after bytes land in the kernel’s volatile page cache (~100ns), while the real disk commit is deferred to background flushers — fast, but a power failure destroys acknowledged data. fsync/fdatasync forces durability to NAND or platters at ~0.5-9ms per call, capping serial commits at hundreds per second. Group commit amortizes one fsync across many concurrent transactions, the trick behind PostgreSQL WAL and Kafka’s page-cache-backed log.
Core Architectural Principles
- Acknowledged-but-dirty transactions define your loss window (RPO) at crash time.
- Commit rate without batching is 1/fsync-latency: NVMe ~2,000/s, HDD ~110/s.
- Group commit divides one disk seek across up to N batched WAL records.
Ground durability claims in mechanics: "a write() ack means DRAM only; for RPO=0 the WAL must fdatasync before acking, so I batch group commits to raise throughput." Mention Kafka relying on the page cache plus zero-copy sendfile() to stream millions of messages per second without JVM heap caching.
Page-cache writes are DRAM-fast but volatile; fsync buys true durability at millisecond cost unless you amortize it with batching.