advanced practice · system design prompt
Production RAG pipeline
Design a RAG system: doc ingestion → chunking → embedding → vector index, plus query-time retrieval → rerank → LLM. Identify the 3 most likely quality regressions.
The mechanism
RAG system design is two pipelines and a contract: an offline ingestion pipeline (chunk → embed → index) and an online query pipeline (retrieve → rerank → pack → generate), bound by an eval harness that gates every change.
Expected evidence
Pipeline + the evals you would run on every change.
Try this drill in the practice workspace →
Hints when you need them
- Chunking dominates retrieval quality more than embeddings do.
- Eval the retriever and the generator separately.
Watch for these mistakes
- No offline/online separation
- No eval gate, so quality regresses silently
- Ignoring freshness and re-indexing