Evaluation & AI Reliability · core
LLM Evals
Measuring LLM output quality with datasets, graders, and LLM-as-judge.
Mental model
Evals are unit tests for non-deterministic systems. Fix a dataset of inputs with expected behavior, define a grader (exact match, rubric, or LLM-as-judge), and run it on every prompt/model change. Scope: this card owns constructing the grader — dataset design, rubric writing, and LLM-as-judge including its calibration and bias checks. Running those graders as a release gate is `ai-regression-testing`; grading an agent's actions rather than its text is `tool-use-evaluations`.
How to study LLM Evals
Begin by restating the mental model in your own words, then connect it to a concrete system you have built or operated. Name the mechanism, the constraint it addresses, and the trade-off it introduces. Use Your AI product needs evals (Hamel Husain), Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, HELM — Holistic Evaluation of Language Models (Stanford CRFM) to check details, but close the source before writing your explanation. Retrieval is the learning step; rereading is only preparation.
Next, compare LLM Evals with Search Evals, Model Routing, Model Evaluation, Hypothesis Testing. Ask what changes in correctness, latency, resource use, operability, and failure recovery. Complete Build an LLM eval and preserve the command, input, output, and one failed attempt as evidence. Finish by explaining the idea without jargon to someone who has not studied the track.
Proof of understanding
- Explain the mechanism from first principles and identify the state it reads or changes.
- Give one situation where the concept is the right choice and one where it is not.
- Predict a realistic failure mode before running the drill, then compare the prediction with evidence.
- Connect the result to a roadmap or build artifact instead of treating the concept as isolated trivia.
Where it matters
Every serious LLM product; regression gates in CI.
Common mistakes
- Shipping prompt changes on vibes with no eval
- LLM-as-judge without checking the judge against humans
- Eval set that does not cover real failure modes
Learn from primary sources
Practice and explain it back
Build an LLM eval
Create a dataset of inputs with expected behavior and a grader (deterministic or LLM-as-judge). Run it as one command and score a prompt.
Expected evidence: A pass-rate score that changes when you change the prompt.
Open the interactive drill →Review prompts
- Why must an LLM-as-judge be validated against human labels?
Build evidence
LLM eval harness
A dataset + grader harness that scores LLM output on every prompt/model change.
- Fixed dataset of inputs with expected behavior
- At least one deterministic grader and one LLM-as-judge grader
- Runs as a single command and prints a score
Model router
Route each request to the cheapest model that passes evals for that request class.
- Classify request difficulty
- Dispatch to a cheap or strong model accordingly
- Track cost and latency per route
Synthesize: Evaluation & AI Reliability
Build an evidence-backed evaluation and observability system for models, tools, and agents. Produce one working system, benchmark, or evidence-backed design that integrates the path.
- Implements or precisely models the core mechanisms from all three milestones
- Includes at least one injected failure or adversarial case and demonstrates recovery
- Reports quality, latency, resource, reliability, or usability measurements relevant to the domain
- Ships a concise architecture note explaining decisions, trade-offs, and remaining risks