Decision Trees
A decision tree is a flowchart of if-then questions the computer builds for itself by playing 20 Questions with the data. This topic covers how CART picks each question, why one tree is wildly unstable, and how pruning and ensembles fix that.
01.The Problem: When "It Depends" Is the Answer
Suppose you predict house prices with a straight line.
Then someone tells you the truth about your market:
- Small houses in the suburbs: families buy them.
- Small houses downtown: investors buy them at very different prices.
- Big houses near schools: everyone fights over them.
The answer is never just "bigger = more." It is "it depends" — the effect of one feature depends on another.
That "it depends" is called an interaction, and a single line (Topic 45: the plain linear model price = b₀ + b₁x₁ + …) literally cannot express it.
You could hand-craft interaction terms.
But with 50 features, how many hand-written combinations do you want to audit — 1,225 pairs, times triples?
There is a second demand too. A hospital, a bank, or a regulator wants to read your model.
"Show me the rule that denied this loan."
A matrix of 50 coefficients does not answer that. A flowchart does.
Decision trees answer both: they find interactions automatically, and they are flowcharts.
Grow, Then Trim: CART Construction ✂️
Grow, Then Trim: CART Construction ✂️
A tree is grown greedily until a stopping rule fires, then pruned back with cost-complexity selection — two independent controls over variance, and neither is learned from the data itself.
Unlock Topic #56: Decision Trees
You are viewing a preview. The full in-depth technical walkthrough, worked derivations, and code notebooks for this concept, along with self-assessment quizzes, are available with Pro or Lifetime Access.
Failure modes, high-throughput bottlenecks, and real FAANG implementation decisions.
Interactive system topology diagrams, live parameter simulators, and downloadable SVG charts.
Staff-level multiple-choice quiz questions with instant feedback and answer explanations.
Firebase Google authentication automatically syncs your completed topics and quiz scores.
How clear and actionable was this distributed systems breakdown?