Because an agent can produce the correct answer for reasons that are wrong, and output accuracy cannot tell the difference. An agent asked whether a customer qualifies calls a lookup (income 80,000), calls the policy (minimum 100,000), and answers yes. If yes is the expected answer, every output check passes while the evidence in the run says the opposite. The agent didn't do its job — it guessed correctly. It fails the other way too: improve an agent past a frozen answer key and the suite books the improvement as a regression.
It isn't randomness or a bad model day. The pattern is that a language model is very good at producing the form of a correct answer, and the form is what an output check inspects. When the underlying reasoning is wrong but the conclusion lands on the expected value, the check has no way to notice.
In a binary decision this happens roughly half the time by chance alone. Any eval suite that only compares final answers on yes/no tasks is awarding partial credit to a coin.
Judge the run. Concretely, four things the output can't tell you:
"Just check the trace" is the obvious conclusion and it's half right. The trap is that the trace is largely the agent's own testimony — the tool stream records what the agent says it called and what it says came back. An agent that will fabricate a conclusion can fabricate the step that supports it.
So evidence provenance decides what you're entitled to conclude. Evidence the agent could not have authored — retrieved source documents, a diff computed independently — can convict. The agent's self-reported tool stream can corroborate, and should not be enough on its own to brand a claim false. Getting this wrong produces false accusations, which cost more than missed ones because they destroy trust in the evaluator itself.
The short answer above is ours, and we've written at length about how we got to it: