Time-Series Gorilla Compression Lab (Interactive)
Encode a real metrics stream bit-by-bit with delta-of-delta and XOR float packing; tune jitter and volatility. Compute live Gorilla-style compressed bytes, savings ratio, and yearly retention for millions of sensor samples per device.
Gorilla Compression & Hypertable Chunk Lab
Encode a real metrics stream with delta-of-delta timestamps and XOR floats; measure the bit-level savings.
Naive row store
31.3 KB
64-bit ts + 64-bit value
Gorilla-encoded
8.0 KB
bit-packed deltas + XOR
Compression saved
74.3%
target band: 90–95%
Hypertable chunks
3
2h chunks; expired ones DROP in O(1)
Delta-of-delta = 0 (1-bit)
37% of timestamps
XOR = 0 (unchanged float)
0% of values
1-year device storage
0.5 MB
naive row store would need 51.7 MB
This is a real bit-level encoder: regular scrape intervals make delta-of-delta zero (1 bit), and slowly-moving sensor values share IEEE-754 prefixes so their XOR collapses to few significant bits. Raise jitter and volatility and watch the 90%+ ratio degrade — exactly why Prometheus pads fixed intervals. Expired chunks are dropped whole (DROP TABLE), never row-by-row DELETE + vacuum.
How It Works Under the Hood
Relational engines collapse under time-series volume because every insert mutates B-tree indexes and deletes fragment tables. TSDBs exploit the append-only, time-ordered invariant instead: Facebook’s Gorilla encodes timestamps as delta-of-deltas, one bit when scrape intervals repeat, and values as XOR against the previous float, near zero when readings barely move. TimescaleDB hypertables chunk time ranges so retention drops whole chunks in O(1), and continuous aggregates downsample raw precision into cheap long-term rollups.
Core Architectural Principles
- Delta-of-delta timestamp coding costs 1 bit for regular scrape intervals and more as jitter rises.
- XOR float encoding stores only significant middle bits of slowly-changing sensor values.
- Chunked hypertables expire retention windows via DROP TABLE, never row-by-row DELETE and vacuum.
For observability or telemetry designs, name the storage mechanics: Prometheus or TimescaleDB with Gorilla compression achieving ninety-plus percent savings, two-hour chunks, continuous aggregates for 1-second raw versus 1-hour rollups. Warn about high-cardinality label explosion, the classic TSDB failure mode, and reserve relational tables for OLTP entities.
Extreme ingest throughput and 90% storage savings versus poor fit for point updates and non-temporal queries.