Recommendation Funnel & ANN Index Lab (Interactive)
Pick HNSW, IVF-PQ, or brute force and watch recall, RAM, and the 50ms four-stage funnel budget move. Sifts a catalog of up to 100M Two-Tower embeddings through retrieval, hard filters, MMoE ranking, and MMR re-ranking, pricing each stage's latency, memory, and recall.
4-Stage Recommendation Funnel & ANN Index
Whittle a 100,000,000-item catalog to a top-20 feed under 50ms: pick the vector index, search effort, and MMR diversity.
Funnel: catalog → top-20 feed
Pipeline total
40.9 ms
within 50ms budget
ANN recall@K
68.5%
fraction of true top-K found
Vector RAM
181.2 GiB
100,000,000 x 256-d vectors, +90% graph links
Feed mix
56 rel / 43 div
balanced like lambda~0.65
Flat search scores all 100,000,000 Two-Tower item embeddings — exact but impossible at 10^8. HNSW descends a layered proximity graph for O(log N) hops at ~1.9x RAM; IVF-PQ quantizes 256-d float32 vectors into codebook bytes, fitting billions on one node at 90–95% recall. The heavy MMoE ranker only ever touches 400 items, and lowering MMR λ penalizes candidates similar to already-selected feed entries.
How It Works Under the Hood
Scoring a 100M-item catalog with a deep ranker per feed refresh is impossible under a 50ms budget, so production recommenders run a funnel: ANN retrieval keeps roughly the top 1,000 candidates, business filters prune to 400, a multi-task MMoE network ranks to 80, and MMR re-ranks to a diverse 20. Each index choice is a memory-recall-latency triangle: Flat is exact but linear, HNSW buys O(log N) graph traversal at ~1.9x RAM, and IVF-PQ compresses vectors ~24-32x for billions-per-node at 90-95% recall.
Core Architectural Principles
- Funnel widths 100M to 1,000 to 400 to 80 to 20 keep the heavy ranker's cost proportional to candidates, not catalog size.
- ANN search effort (ef_search or nprobe) trades recall and latency; RAM footprint differs 1x/1.9x/0.04x per index type.
- MMR lambda penalizes similarity to already-selected items, moving the feed between relevance and diversity.
Sketch the four-stage funnel first and assign explicit latency budgets (8ms retrieval, 25ms ranking, 10ms re-rank, 5ms network). Explain why Two-Tower item embeddings precompute into an HNSW index while the user tower runs online. Then justify MMoE over raw CTR to avoid clickbait, and MMR plus exploration bandits for filter bubbles and cold start. Quantify index RAM to show you sized the vector database.
Approximate indexes and narrow funnel stages buy sub-50ms latency and affordable RAM, but sacrifice exact recall, while aggressive business re-ranking can override good ML scores.