Random Forests
One decision tree is a moody expert: ask it twice on slightly different data and it changes its mind. A random forest grows hundreds of deliberately different trees and lets them vote — averaging cancels their individual mood swings and yields one of the most robust general-purpose tabular learners.
01.The Problem: One Tree Changes Its Mind Too Easily
Recall the single decision tree (Topic 56: a flowchart of if-then questions, grown greedily by picking the best split at every node).
It has an embarrassing flaw: instability.
Regrow the same tree on 90% of your data and the root question itself can change. Different root → different boxes everywhere below → a totally different flowchart.
Think of a moody expert:
- Ask them on Monday: "small house, suburbs — worth what?" → one answer.
- Reshuffle their notes a little, ask Tuesday → a different answer.
Statistically: a fully grown tree is high variance, low bias (Topic 47: variance = predictions swing wildly with the training sample; bias = systematically off).
Can we train a stable single tree?
Not really — stability is baked into greedy partitioning.
So what do we do with a moody but well-meaning expert?
Hire a lot of them and average their opinions. That is the entire idea of the random forest.
Random Forest = Bagging + Feature Randomization 🌳
Random Forest = Bagging + Feature Randomization 🌳
Each tree sees a bootstrap sample AND can only consider a random feature subset at every split, decorrelating errors so averaging actually helps.
Unlock Topic #59: Random Forests
You are viewing a preview. The full in-depth technical walkthrough, worked derivations, and code notebooks for this concept, along with self-assessment quizzes, are available with Pro or Lifetime Access.
Failure modes, high-throughput bottlenecks, and real FAANG implementation decisions.
Interactive system topology diagrams, live parameter simulators, and downloadable SVG charts.
Staff-level multiple-choice quiz questions with instant feedback and answer explanations.
Firebase Google authentication automatically syncs your completed topics and quiz scores.
How clear and actionable was this distributed systems breakdown?