TOPIC #79Intermediate 7 min read

Read-Your-Writes & Session Consistency

CSD
CompleteSystemDesign Editorial
Report an issue
Key takeawayCore Architecture Summary

Prevent user confusion in eventually consistent systems: Sticky routing, version timestamps, and client-side session tokens.

Key Glossary Concepts in this TopicAll Glossary Terms
Interactive Lab · 🍪 Session Consistency RouterFull lab guide

Read-Your-Writes Session Router

10 users update their profile, then reload 60ms–3s later against lagging replicas. Pick a session strategy and count the “my edit vanished” bugs.

p99 replication lag500 ms
Sticky-pin window (3× lag heuristic)1500 ms
Applies to the time-based primary pin strategy. Set it near 3× p99 lag.
Stale reads
6/10
Reads on primary
0/10
Replica offload
100%
Session ledger (write LSN vs served LSN)
user 1 · read +100ms · replica 1LSN 1000/1001 ✗ STALE
user 2 · read +400ms · replica 2LSN 1001/1002 ✗ STALE
user 3 · read +800ms · replica 0LSN 1003/1003 ✓ fresh
user 4 · read +200ms · replica 1LSN 1003/1004 ✗ STALE
user 5 · read +1500ms · replica 2LSN 1005/1005 ✓ fresh
user 6 · read +500ms · replica 0LSN 1005/1006 ✗ STALE
user 7 · read +90ms · replica 1LSN 1006/1007 ✗ STALE
user 8 · read +3000ms · replica 2LSN 1008/1008 ✓ fresh
user 9 · read +2000ms · replica 0LSN 1009/1009 ✓ fresh
user 10 · read +60ms · replica 1LSN 1009/1010 ✗ STALE

Naive round-robin: Naive balancing is the anomaly generator: successive reads from one session land on differently-lagged replicas, producing stale reads and even monotonic-read time travel.

Read-Your-Writes Anomaly vs Resolution 🔄

User submits a profile update but reads from a lagging replica.

Read-Your-Writes Anomaly vs Resolution 🔄
100%
Touchpad: Pinch to zoom • Drag to pan
Rendering visual architecture flowchart...

01.The Read-Your-Writes Anomaly (The User-Experience Disaster)

In modern web architectures utilizing asynchronous primary-replica replication or eventually consistent distributed databases, replication lag is normal (50ms to 500ms).

However, this creates severe user experience bugs:

  1. A user updates their profile picture or comments on a forum post.
  2. The write is committed to the Primary Database, returning an HTTP 200 OK.
  3. The client application immediately triggers a page re-render, dispatching a GET request.
  4. The Load Balancer routes the read request to an asynchronous Read Replica that is 200ms behind the primary.
  5. The replica returns the old profile picture or shows that the comment does not exist.
  6. The user assumes their action failed, clicks the submit button multiple times, and files a bug report.

Read-Your-Writes Consistency (Session Consistency) guarantees that a given user session will always immediately observe the effects of their own updates, even if other users on the platform observe them with slight replication delay.

02.Implementation Patterns for Read-Your-Writes Consistency

System designers implement session consistency using three primary architectural patterns:

1. Write Log Sequence Number (LSN) / Version Tracking (Gold Standard)

  • When the primary database commits a write, it returns the Log Sequence Number (LSN) or transaction version timestamp (T_{commit}).
  • The API Gateway passes this metadata back to the client browser in an HTTP response header or session cookie: Set-Cookie: min_lsn=894321; HttpOnly.
  • On subsequent read queries, the client forwards the cookie. The load balancer / proxy inspects available read replicas:
    • If (replica.current_lsn \ge min_lsn), the read routes to that replica.
    • If all replicas are lagging behind (min_lsn), the proxy either waits briefly (< 20ms) or falls back to querying the Primary database.

2. Time-Based Primary Sticky Pinning (Heuristic Approach)

  • Whenever a user performs a write operation, set an ephemeral session flag (in Redis or an encrypted cookie) with a TTL equal to 3 × p99 replication lag (e.g., 5 seconds).
  • For the next 5 seconds, all read queries from that specific user session are pinned directly to the Primary Database.
  • After 5 seconds, reads smoothly revert back to the Read Replica pool.

3. User-ID Affinity Routing

  • Partition users consistently across replica subsets using consistent hashing on (hash(user_id)). If writes to that user's partition stream to a dedicated local replica, the client always reads from the replica receiving their stream.

03.Monotonic Reads & Monotonic Writes Guarantees

Read-Your-Writes is part of the four classical Client-Centric Consistency Guarantees:

  1. Read-Your-Writes: A user's successive reads will always reflect their own previous writes.
  2. Monotonic Reads: If a user reads version v_2 of a record, all subsequent reads by that user will return version v_2 or newer—they will never "time-travel" backwards to observe an older version v_1 (which happens if successive reads hit differently lagging replicas).
  3. Monotonic Writes: A user's successive write operations are guaranteed to be serialized and executed in the exact order they were submitted.
  4. Writes-Follow-Reads (Causal Writes): If a user writes an update after reading version v_1, that new update is guaranteed to causally take effect after v_1.

Architectural Trade-offs & Production Realities

Architectural Advantages

  • Completely eliminates user confusion and duplicate form submissions caused by replication lag
  • Preserves 90%+ offloading of read traffic to horizontal read replicas for browsing users

Trade-offs & Constraints

  • Adds routing state metadata in cookies/headers and requires database proxy or gateway awareness
  • Heavy-writing users will frequently route reads to the primary database, slightly reducing replica offloading
Production Implementation in Big Tech
Facebook (Meta)• TAO Cache Read-After-Write

When a user posts a status on Facebook, TAO routes subsequent reads from that specific user to their local leader master cache to guarantee immediate visibility.

Staff+ Engineering Takeaways

  • Read-Your-Writes guarantees a client always sees their own recent updates.
  • Achievable via LSN cookies, read-after-write primary pinning, or monotonic session tokens.
  • Monotonic reads prevent time-travel anomalies where subsequent requests hit lagging replicas.
  • Protects user trust without needing global distributed locks.

Topic Knowledge Check

Exercise 1 of 1 • Test your architectural comprehension.

Exercise 1 of 10 answered
1

How can an API Gateway enforce Read-Your-Writes without routing all reads to the primary database?

Rate This Architecture ChapterFeedback & Rating

How clear and actionable was this distributed systems breakdown?

Interactive Engineering Workbenches: