Transfer Learning
You have 2,000 images and no hope of training a big network from scratch. The fix: start from a network that already studied 1.28 million — its early layers (edges, textures) work for any image, so you only train the part that knows YOUR classes. This topic covers feature extraction vs fine-tuning, freezing, discriminative learning rates, and modern self-supervised / vision-language pretraining.
01.The Problem: You Do Not Have a Million Pictures
Suppose you want an app that tells 120 dog breeds apart.
You collect 2,000 photos. Roughly a few hundred per common breed, a dozen or two per rare one. That is your data.
Now try the naive plan: build a ResNet-style convolutional network (a deep stack of sliding filters — the classic image architecture) and train it from scratch — weights filled with random noise, learning everything from your 2,000 images.
What happens? The network memorizes your pictures instead of learning the breeds. Training accuracy hits 100%; accuracy on new photos face-plants. That is overfitting: with only 2,000 examples and millions of knobs to tune, the model finds a cheat code that works only on the exact images it saw.
So the question becomes
Where do you get the "understanding of images" that 2,000 photos cannot teach?
Answer: borrow it. Someone already trained a network on 1.28 million labeled photos (ImageNet). It had to learn edges, textures, shapes, and objects just to survive. You can start from its weights instead of random noise.
That borrowing is transfer learning, and it is the default workflow of applied computer vision — not a corner case.
Reusing a pretrained backbone 🔁
Reusing a pretrained backbone 🔁
Early layers (green) generalize across tasks; the final ImageNet head (red) is task-specific and gets replaced. The frozen-vs-fine-tuned decision governs the backbone weights.
Unlock Topic #105: Transfer Learning
You are viewing a preview. The full in-depth technical walkthrough, worked derivations, and code notebooks for this concept, along with self-assessment quizzes, are available with Pro or Lifetime Access.
Failure modes, high-throughput bottlenecks, and real FAANG implementation decisions.
Interactive system topology diagrams, live parameter simulators, and downloadable SVG charts.
Staff-level multiple-choice quiz questions with instant feedback and answer explanations.
Firebase Google authentication automatically syncs your completed topics and quiz scores.
How clear and actionable was this distributed systems breakdown?