Vector DB & ANN · advanced
IVF (Inverted File)
Cluster vectors with k-means, then search only the nearest cells (nprobe).
Mental model
IVF partitions the space into Voronoi cells via k-means. A query is matched to its nearest cells (nprobe of them) and only those vectors are scanned — trading recall for a big speedup.
How to study IVF (Inverted File)
Begin by restating the mental model in your own words, then connect it to a concrete system you have built or operated. Name the mechanism, the constraint it addresses, and the trade-off it introduces. Use Pinecone — IVF index to check details, but close the source before writing your explanation. Retrieval is the learning step; rereading is only preparation.
Next, compare IVF (Inverted File) with HNSW, Product Quantization. Ask what changes in correctness, latency, resource use, operability, and failure recovery. Complete IVF nprobe vs recall and preserve the command, input, output, and one failed attempt as evidence. Finish by explaining the idea without jargon to someone who has not studied the track.
Proof of understanding
- Explain the mechanism from first principles and identify the state it reads or changes.
- Give one situation where the concept is the right choice and one where it is not.
- Predict a realistic failure mode before running the drill, then compare the prediction with evidence.
- Connect the result to a roadmap or build artifact instead of treating the concept as isolated trivia.
Where it matters
FAISS IVF indexes, often combined with product quantization (IVF-PQ).
Common mistakes
- Setting nprobe=1 and wondering why recall is poor
- Too few or too many centroids for the dataset size
Learn from primary sources
- Pinecone — IVF index (article)
Practice and explain it back
IVF nprobe vs recall
100k vectors, 1000 centroids, nprobe=1 recalls 70%, nprobe=8 recalls 92%. Latency 2ms→9ms. Pick nprobe for prod if SLA is 5ms and recall target 85%.
Expected evidence: nprobe=8 exceeds recall but misses SLA; try nprobe=4 and measure curve.
Open the interactive drill →Review prompts
- What happens to recall and latency as you increase nprobe in IVF?
Build evidence
Use a roadmap capstone to turn this concept into working evidence.