Encoding Categorical Variables
Models read numbers, not words like "red" or "Delhi". Encoding converts categories into numbers without smuggling in fake relationships: one-hot for unordered labels, ordinal for genuinely ranked ones, and target/embedding tricks for columns with thousands of categories.
01.The Problem: Models Cannot Read "Red"
Your dataset has a column color with values red, green, blue.
You hand it to a linear model and it errors out — or worse, silently does nothing useful. Why? Most algorithms accept only numeric input. Words are not numbers.
So the question becomes
How do I turn labels into numbers without lying to the model?
Here is the trap. The lazy answer is "just count them":
- red → 1, green → 2, blue → 3
Done? No. You just told the model something false. Numbers come with built-in assumptions:
- 3 > 2 > 1 → "blue is greater than red"
- distance(1, 3) = 2 × distance(1, 2) → "blue is twice as far from red as green is"
None of that is true. Red, green and blue are just names — categories with no order. Making up an order teaches the model nonsense; a linear model might learn "blue is three times red".
The whole point of encoding:
Represent categories numerically without smuggling in false relationships.
Choosing an Encoding
Choosing an Encoding
Nominal categories go one-hot; ordered categories go ordinal; and very high-cardinality columns use target or embedding encoding to avoid explosion.
Unlock Topic #33: Encoding Categorical Variables
You are viewing a preview. The full in-depth technical walkthrough, worked derivations, and code notebooks for this concept, along with self-assessment quizzes, are available with Pro or Lifetime Access.
Failure modes, high-throughput bottlenecks, and real FAANG implementation decisions.
Interactive system topology diagrams, live parameter simulators, and downloadable SVG charts.
Staff-level multiple-choice quiz questions with instant feedback and answer explanations.
Firebase Google authentication automatically syncs your completed topics and quiz scores.
How clear and actionable was this distributed systems breakdown?