Normalization vs Standardization
Age runs 0–100, income runs 0–1,000,000 — on one ruler, income shouts and age whispers. Scaling fixes that. Min-Max stretches every value into [0,1]; Z-score standardization recenters to mean 0 and spread 1. This topic shows both formulas, tiny worked examples, and when to pick each.
01.The Problem: One Column Shouts, the Other Whispers
You hand a model two columns:
- age: 25, 30, 45 — numbers in the tens
- income: 30,000, 80,000, 900,000 — numbers in the tens of thousands
Both are "just numbers", right?
So why does the model suddenly treat income as everything and age as nothing?
Because many models measure distance between rows. In Euclidean distance, a difference of 50,000 in income dwarfs a difference of 5 in age — the unscaled income column simply dominates every calculation.
The same trap hits gradient descent (Topic 49): features with huge magnitudes produce tiny, unstable gradients for the other weights, and training crawls or zig-zags.
Scaling fixes it: rescale every column so all features stand on a comparable footing, and no single column unfairly drives the result.
One exception worth knowing early: tree-based models (random forest, gradient boosting) split on thresholds — "is income > 50,000?" — and a threshold question gives the same answer no matter what unit the numbers are in. They are scale-invariant and do not require scaling.
Two Ways to Scale a Feature
Two Ways to Scale a Feature
Normalization compresses values into a fixed [0,1] range; standardization recenters to zero mean and unit variance and is more robust to outliers.
Unlock Topic #32: Normalization vs Standardization
You are viewing a preview. The full in-depth technical walkthrough, worked derivations, and code notebooks for this concept, along with self-assessment quizzes, are available with Pro or Lifetime Access.
Failure modes, high-throughput bottlenecks, and real FAANG implementation decisions.
Interactive system topology diagrams, live parameter simulators, and downloadable SVG charts.
Staff-level multiple-choice quiz questions with instant feedback and answer explanations.
Firebase Google authentication automatically syncs your completed topics and quiz scores.
How clear and actionable was this distributed systems breakdown?