ETL vs ELT Pipeline Lab (Interactive)
Trigger metric redefinitions and watch ELT recompute history while legacy ETL loses raw data forever. Compare storage bills, backfill cost, and historical metric fidelity between external-transform ETL and dbt-powered in-warehouse ELT with a Bronze raw tier.
Legacy ETL vs Modern ELT Pipeline
Price storage, backfills, and metric fidelity when business definitions inevitably change.
How It Works Under the Hood
Legacy ETL transformed data on a dedicated server and loaded only aggregates, discarding raw rows because warehouse disk was expensive. When a definition like churn changed, history was unreconstructible. Modern ELT inverts this: object storage is effectively cheap, so immutable raw JSON lands in a Bronze tier and dbt SQL models build Silver and Gold inside elastic warehouse compute. This simulator prices the storage amplification of keeping raw against the backfill cost per redefinition, exposing why preserving history is the core architectural insurance of the modern data stack.
Core Architectural Principles
- Bronze raw tier plus dbt-built Silver/Gold yields roughly 1.6x storage amplification versus aggregates-only ETL.
- Sources keep change history ~90 days, so ETL cannot rebuild metrics older than that window.
- dbt incremental materializations recompute only new watermark-sliced records instead of full scans.
Advocate ELT with dbt as the modern default and name the medallion tiers. Then show the failure mode you are avoiding: in ETL, a changed active-user definition silently corrupts every historical dashboard because raw rows were discarded. Quantify backfill as a dbt re-run over Bronze at warehouse scan cost, and mention CDC via Debezium for zero-lock ingestion from OLTP databases.
Raw immutability grants retroactive correctness but inflates storage footprints and forces strict PII masking policies.