Data Compression & Protobuf Sizing Lab (Interactive)
Pick Snappy, Gzip, Zstd, or Brotli, switch JSON to Protobuf, and flip into the pre-compressed-media and tiny-payload traps. Compute wire size, monthly egress bills at $0.09/GB, and compression CPU core-hours for each algorithm and serialization choice.
Compression & Wire-Format Bill Calculator
Trade CPU cycles for egress dollars — and fall into the traps that invert the trade.
How It Works Under the Hood
Compression trades cheap CPU cycles for expensive bandwidth: cloud egress at $0.08-0.12/GB means 70% compression on a daily petabyte saves tens of thousands of dollars monthly. Each tool owns a niche — Snappy/LZ4 near-zero-overhead streaming, Zstd tunable levels with constant gigabyte-per-second decompression for data lakes, Brotli maximum density for static web assets. Serialization is the pre-compression lever: Protobuf strips repeated field names and varint-encodes integers, shrinking JSON 60-80% before any compressor runs.
Core Architectural Principles
- Wire-size chain: 1,000-byte JSON becomes 280 Gzipped, 120 as Protobuf, 38 as Protobuf plus Zstd.
- Snappy compresses 500-800 MB/s at ~45% ratio; Brotli hits ~22% ratio but roughly 25 MB/s — choose per path.
- Traps invert the trade: Gzip over JPEG grows files, and sub-1 KB responses pay Huffman-header overhead.
Recommend per surface, not per preference: Brotli for static HTTPS assets, Zstd for Kafka and Parquet, Snappy when CPU is the bottleneck, Protobuf for internal gRPC. Then quote gzip_min_length 1024 and never-compress-media — those two anti-pattern call-outs prove you have shipped this, not just read about it.
Every compression cycle is CPU you pay on both ends to buy bandwidth and storage you rent by the gigabyte, and the trade turns negative on already-entropied or tiny payloads.