An AI agent is ready for production when three things are true: someone has written down what success means in terms specific enough to check, there is evidence each of those conditions was met on real runs, and a named person is willing to sign off on that evidence. A passing test suite is not the same thing. Tests tell you the agent behaved on the cases you thought of; readiness is a claim about whether you can rely on it, including on the cases you didn't. Most teams have the first kind of evidence and are being asked for the second.
Drift harnesses and regression suites are good engineering, and worth building. They answer a specific question well: did the agent behave the way it behaved last week? That catches real problems.
But the question a stakeholder has to put their name on is different: can we rely on this? Those come apart in ways that matter. An agent can pass every test and still fabricate a citation on an input nobody wrote a test for. An agent can fail a test because it got better — a golden answer key goes stale, the agent returns the now-correct answer, and the suite marks it a regression.
The gap isn't a gap in test coverage. It's that a test compares output to an expectation, and readiness is a question about properties: did it stay inside policy, was every claim grounded in something real, did it miss anything that mattered, what did it cost.
Concretely, before an agent ships you want to be able to hand someone:
If you want a single diagnostic, ask the team this: if it passed every test you have, would you ship it tomorrow?
Hesitation is the answer. It means the tests aren't measuring the thing that would let someone sign off, and everyone in the room already knows it. That gap is what readiness work is actually for.
The obvious shortcut is to have a strong model read the transcript and judge whether things went well. It works better than people expect on coarse questions and worse than people expect on precise ones, and the difference is not about model strength.
A judge is reliable in proportion to how well-posed the question is. Ask "which of these agents failed" and a frontier model does fine. Ask "which single step was the decisive error" — a question competent human reviewers answer differently from each other — and accuracy collapses, and a better model doesn't rescue it. The fix is making the question determinate before asking, which is what a written contract does.
The short answer above is ours, and we've written at length about how we got to it: