Instagram Logical Shard ID Lab (Interactive)
Bit-pack a 64-bit media ID from timestamp, shard, and sequence, then route it back to a schema. Encode Instagram-style logical IDs by hand: pack a 41-bit timestamp, 13-bit shard ID, and 10-bit sequence into one 64-bit integer, parse the bits to find the PostgreSQL schema, and project shard growth.
Instagram Shard-ID Packer & Growth Planner
Build a 64-bit self-routing photo ID from timestamp, logical shard, and sequence bits, then plan PostgreSQL schema growth across physical boxes.
How It Works Under the Hood
Instagram grew from a single PostgreSQL database to logical sharding without downtime. Every Media object ID is a 64-bit number whose bits encode a 41-millisecond timestamp, a 13-bit shard identifier, and a 10-bit per-shard sequence. IDs stay self-routing — parsing the shard bits tells the data access layer which schema owns the row — while remaining monotonic within each shard for index locality. Shards are added in pairs, and the logical tier between Django models and MySQL keeps billions of photos, edges, and counters sharding cleanly across thousands of schemas.
Core Architectural Principles
- Bit packing: (timestamp << 23) | (shard << 10) | sequence builds a routing-capable, time-sortable 64-bit ID.
- Self-routing lookups: unpacking the shard bits locates the owning PostgreSQL schema with no lookup table.
- Growth planning: schemas per shard and rows per year expose the hottest-shard skew Instagram budgets at roughly 2.5x average.
In data-modeling rounds, contrast three shard-key strategies: hash of ID (even, no range scans), range (time locality, hot shards), and Instagram's logical ID that embeds the shard so any ID routes itself. Showing you can hand-count the bit layout — 41 plus 13 plus 10 equals 64 — signals real understanding instead of buzzword recall.
Logical IDs give deterministic routing and index locality but burn 23 of 64 bits on shard and sequence, capping shards at 8,192 and sequences at 1,024 per millisecond.