90d roadmap · 4 milestones
90-Day AI Search & Infrastructure
The core roadmap. Three months that move from retrieval basics to ANN, RAG, and the storage foundations behind systems like Turbopuffer.
Outcome: Build a strong retrieval, vector, AI-systems, and storage foundation through HighSignal and Codevetter.
How to work this roadmap
Treat each milestone as a claim that must be supported by evidence. Before reading, write what you think the mechanism does and where it will fail. After studying the linked concepts, run the drills without copying an answer, preserve the output, and revise the explanation. Move forward when you can connect the milestone goal to a working implementation, benchmark, architecture decision, or reviewable design artifact.
The 90d horizon is a sequencing aid, not a completion badge. Spend more time where your prediction and the observed behavior disagree. Keep a short decision log containing the mechanism selected, alternatives rejected, expected failure mode, measurement used, and remaining uncertainty. Review that log with FSRS prompts so the roadmap produces durable system judgment rather than a temporary tour of terminology.
At the end, explain Build a strong retrieval, vector, AI-systems, and storage foundation through HighSignal and Codevetter. from first principles to a reader outside the domain. A strong explanation should survive follow-up questions about correctness, cost, latency, resource use, security, recovery, and operational visibility. If it cannot, return to the milestone that contains the missing mechanism and build a smaller falsifiable example.
Milestones
Milestone 1
Prerequisites — unlock the rest of this path
Concepts the later milestones depend on. Without these the planner cannot serve the rest of this roadmap.
Concepts
- Learning to Rank
- Scalar & Binary Quantization
- Disk-Based ANN
- Index Updates & Tombstones
- Top-k Pruning (WAND)
- Descriptive Statistics
- Probability Fundamentals
- Random Variables & Distributions
- Ranking Metrics
- Structured Outputs
- TF-IDF
- Tokenization
- Vector Similarity
- Estimation & Confidence Intervals
- Hypothesis Testing
- Top-k Vector Search
Milestone 2
Month 1 — Retrieval and evaluation
Lexical + vector retrieval, hybrid search, evals.
Concepts
Build evidence
- Implement BM25 search in HighSignal — Index a corpus of articles and serve ranked keyword search using BM25.
- Hybrid search v0 in HighSignal — Combine BM25 and vector retrieval with reciprocal rank fusion.
- Search eval harness — A reusable harness that scores any retriever against a labelled query set.
Milestone 3
Month 2 — Vector databases and ANN
Brute force, HNSW, filtering, recall/latency tradeoffs.
Concepts
Build evidence
- Brute-force vector index — An exact nearest-neighbour index — the recall=1.0 baseline.
- Metadata filtering for vector search — Add structured predicate filtering to vector search.
- Recall / latency benchmark dashboard — Plot recall vs latency for ANN parameter sweeps against the brute-force baseline.
Milestone 4
Month 3 — AI systems and storage
RAG with evals, plus the storage foundations underneath.
Concepts
Build evidence
- RAG pipeline v0 — An end-to-end retrieve → pack → generate pipeline grounded in a document corpus.
- LLM eval harness — A dataset + grader harness that scores LLM output on every prompt/model change.
- Toy write-ahead log — An append-only WAL with crash recovery.
- Object-storage-backed index — Store index segments in object storage with a hot in-memory cache.