Agent Systems · advanced
Independent Agent Verification
Maker-checker role separation, independent evidence gathering, adversarial review, and acceptance gates that do not trust the producer's self-report.
Mental model
The maker proposes a change; the checker reconstructs the contract and gathers fresh evidence. Independence means the checker does not merely repeat the maker's reasoning or accept its claimed test result. Release requires both an artifact and externally reproducible evidence against explicit criteria.
How to study Independent Agent Verification
Begin by restating the mental model in your own words, then connect it to a concrete system you have built or operated. Name the mechanism, the constraint it addresses, and the trade-off it introduces. Use Anthropic — Harness design for long-running application development, OpenAI — Harness engineering to check details, but close the source before writing your explanation. Retrieval is the learning step; rereading is only preparation.
Next, compare Independent Agent Verification with Agent Observability, Coding Agent Systems, Harness Evaluation. Ask what changes in correctness, latency, resource use, operability, and failure recovery. Complete Separate maker claims from checker evidence and preserve the command, input, output, and one failed attempt as evidence. Finish by explaining the idea without jargon to someone who has not studied the track.
Proof of understanding
- Explain the mechanism from first principles and identify the state it reads or changes.
- Give one situation where the concept is the right choice and one where it is not.
- Predict a realistic failure mode before running the drill, then compare the prediction with evidence.
- Connect the result to a roadmap or build artifact instead of treating the concept as isolated trivia.
Where it matters
Agent-generated pull requests, evaluator agents, security review, visual regression, acceptance tests, and policy gates.
Common mistakes
- Asking the same context to review its own work without fresh evidence
- Using lint success as proof of user-facing or semantic correctness
- Letting a checker rewrite the work before clearly reporting the failed acceptance criterion
Learn from primary sources
Practice and explain it back
Separate maker claims from checker evidence
Implement checkerVerdict(contract, makerReport, observations). contract contains requiredChecks and maxChangedFiles. observations contains independently observed checks and changedFiles. Return {accepted, missing, reasons}. Acceptance must depend only on the contract and observations, never on makerReport.claimedPassed. A required check passes only when observations records it as 'passed'.
Expected evidence: An independent acceptance verdict that cannot be fooled by a producer's unsupported claim.
Open the interactive drill →Review prompts
- A maker agent says it ran all tests and asks a checker agent to approve the change. What makes the check independent rather than another self-review?
Build evidence
Build a Complete Coding-Agent Harness
Build a repository-local harness that can take one bounded engineering task from initialization through implementation, verification, durable handoff, and explicit termination without relying on hidden machine state or chat memory.
- A fresh isolated workspace can bootstrap and prove readiness from repository-local commands
- Instruction scopes, allowed edit roots, capabilities, budgets, and escalation triggers are explicit and mechanically inspectable
- The run persists goal, progress, evidence, blockers, and the next safe action across a forced restart
- A checker gathers independent evidence and can reject an unsupported maker claim
- The lifecycle terminates only as complete, blocked, failed, cancelled, or escalated with durable evidence
Automate and Evaluate a Maker-Checker Loop
Automate a bounded loop in which a maker attempts one task, an independent checker evaluates explicit acceptance criteria, failures return actionable evidence, and the orchestrator stops on success, escalation, cancellation, or budget exhaustion. Compare it with a baseline harness on a frozen task suite.
- Maker and checker roles have separate inputs and the checker ignores unsupported self-reported success
- The loop has explicit goal, task and retry budgets, durable state, and semantic termination criteria
- A frozen evaluation suite measures success, unsafe actions, cost, latency, retries, and human interventions
- At least one component ablation isolates whether instructions, tools, verification, or retry policy caused the observed change
- Release or rejection follows a declared multi-objective gate rather than pass rate alone