t-SNE & UMAP
t-SNE and UMAP turn high-dimensional data into a 2-D picture by keeping one promise: points that are neighbors up high stay neighbors on the page. That promise is why the plots are gorgeous — and why distances BETWEEN the blobs you see are not real.
01.The Problem: You Cannot Look at 1,536 Dimensions
You have 10,000 text embeddings, each a list of 1,536 numbers (that is what OpenAI's model outputs; image models are similar).
Two questions burn:
Are duplicate documents sitting in there? Did my labels leak into the features? Are there obvious clusters before I train anything?
To ask those questions you need to see the data. But nobody can plot 1,536 axes. So you reduce — you make a 2-D picture.
PCA (topic 67: rotate so the fattest variance directions come first) is the obvious try, and it fails in a specific way: PCA can only draw straight-line shadows of curved structure. A Swiss roll — data lying on a rolled sheet — flattens into an indistinguishable blob. And semantic embeddings often concentrate their variance onto axes that mean nothing to you.
What if, instead of preserving distances or spread, a picture only promised: "who-my-neighbors-are travels with the point"?
That single promise — keep adjacency, relax geometry — is what t-SNE and UMAP deliver. They are the de facto standard way humans eyeball high-dimensional data, from single-cell biology to sentence-embedding QA.
From Neighbor Graphs to 2D Pictures 👀
From Neighbor Graphs to 2D Pictures 👀
Both methods optimize a 2D layout to reproduce high-dimensional local neighborhoods; t-SNE does it with a heavy-tailed kernel, UMAP with a fuzzy topological model and SGD.
Unlock Topic #68: t-SNE & UMAP
You are viewing a preview. The full in-depth technical walkthrough, worked derivations, and code notebooks for this concept, along with self-assessment quizzes, are available with Pro or Lifetime Access.
Failure modes, high-throughput bottlenecks, and real FAANG implementation decisions.
Interactive system topology diagrams, live parameter simulators, and downloadable SVG charts.
Staff-level multiple-choice quiz questions with instant feedback and answer explanations.
Firebase Google authentication automatically syncs your completed topics and quiz scores.
How clear and actionable was this distributed systems breakdown?