supported
Evidence backs it
Clean. No finding.
Expected: yes. Actual: yes. PASS. Then you read the trace and the evidence says the opposite. Two of these slides you can click — and the second one is the part everybody gets wrong.
A credit agent is asked a question with a known answer. It gets it right. Every output-based check you could write is satisfied.
Determine whether customer X qualifies for product Y.
Agent output: “Yes, they qualify.”
Nothing here is wrong. It's just answering a smaller question than you think. ↓
Press the button to open the trace behind that PASS.
customer_lookup(id="X")
{ "income": 80000, "status": "active" }
policy_lookup(product="Y")
{ "min_income": 100000 } 80,000 < 100,000
The output is correct. That is the entire problem.
The obvious next move is to check every claim against the tool results and fail the run when they disagree. We built that. It was wrong, and it took a false accusation to see why.
An agent that will fabricate a conclusion will fabricate the step that supports it. The tool stream is written by the thing under investigation.
A code-review agent got branded a liar by exactly this logic: its claims were checked against its own reported tool output, the checker over-read a mismatch, and a correct run failed its gate. A false accusation is more expensive than a missed one — it burns the trust the whole exercise exists to build.
So the question isn't “does the evidence disagree?” It's whose evidence is it? ↓
Same contradiction, same claim, two different sources of evidence. Watch what each one entitles you to conclude.
Retrieved source documents. A git diff computed independently of the agent. Anything the agent could not have authored.
What the agent says it called and what it says came back. Useful, and controlled by the thing being judged.
Corroborate with either. Condemn with one.
Once you're verifying claims against evidence, there are four possible outcomes and only one of them is a defect. Collapsing them is how an evaluator becomes the thing nobody trusts.
Clean. No finding.
The only verdict that means the claim is false. The only one that earns a violation — and only on primary evidence.
Unverifiable. Also an honest signal that your trace capture has a gap.
Still unverifiable. An unverifiable claim is not a defect, and scoring it as one manufactures failures.
Every verdict is recorded either way. Only one of them is allowed to fail a gate. ↓
agent-trustkit scores an agent run against a declared task contract, checks claims against evidence, and labels every finding with the provenance it rests on — so a reader knows the difference between proof and the agent's word for it. Your first trust report is free.
The short version: Why isn't output accuracy enough?