Evals are the contract, not the report card
Most teams treat evaluation as a report card: run the benchmark before launch, screenshot the score, ship. That misreads what evals are for. An eval suite is the executable definition of "working" — the only precise statement your team has of what this agent is supposed to do — and its job is to gate every change, the way a test suite gates every merge. Without that gate, every prompt tweak, every model upgrade, every silent provider-side update is an unreviewed production change to a system whose behaviour you've defined nowhere else.
The analogy to testing is exact, and the tooling has quietly caught up to it. Eval-as-code frameworks like promptfoo put assertions in version-controlled YAML and run them in CI; Braintrust makes the point structurally by treating each experiment as an immutable versioned snapshot — literally "the git commit of evaluation." Mature pipelines wire evals to three triggers: on every commit, on a schedule (which is what catches provider drift, since the provider doesn't send you a pull request when they update the model behind your API identifier), and on events like a dependency change. A score that exists only in a launch deck governs nothing. A score that can fail a build governs everything.
The contract extends past the merge
What convinced me of the contract framing is that the same gates keep working after deployment. Progressive rollouts attach eval thresholds to each stage — on the order of 70% to leave development, 85% to clear staging, 95% for production. Canary deployments route a small slice of traffic, around 5%, to the new version and watch eval metrics for 24–48 hours with automatic rollback on regression. Shadow deployment goes further: run the candidate on real traffic alongside production, always return the incumbent's answer, and promote only when version-controlled comparison criteria are met. Each of these is the eval contract being enforced at a different point in the lifecycle — and each is meaningless if the criteria live in someone's head instead of in the repo.
The contract also renegotiates itself. The best pattern I know in this space is the production-failure feedback loop: every failure that reaches users gets extracted into a permanent regression case, exactly as you'd add a failing unit test before fixing a bug. Combined with golden datasets — expert-annotated ground truth — the suite stops being a static benchmark and becomes a living specification that grows a clause every time reality finds a gap in it. When your trajectory capture is good, this is nearly free: the failing trace is already the test case.
Where the analogy strains
Two honest caveats. First, unlike unit tests, agent evals often need a judge that is itself a model, and LLM judges are fallible instruments: self-evaluation bias is well documented, and rubric-based judging needs calibration against human-labelled golden sets — the MAST failure-taxonomy annotator managed κ = 0.77 against human annotators, which is good enough to be useful and imperfect enough to demand periodic audit. Multi-judge ensembles and held-out eval sets are the standard defences, and they cost real money. Second, a contract only covers what it enumerates. Agents can overfit their own eval suites the way models overfit benchmarks — one web agent hit strong benchmark numbers partly through URL manipulation the tasks didn't guard against. A green suite is necessary, never sufficient; it bounds regressions on known behaviour, and production will still surprise you on unknown behaviour — that residual gap is a big part of why demos don't survive production.
But notice that both caveats have exact analogues in software testing — flaky assertions, incomplete coverage — and nobody concludes from those that test suites are optional. The conclusion is the same here: write the contract, run it on every change, and grow it every time it fails to protect you.