As four separate questions, because they fail for different reasons and have different fixes. Was the tool permitted — mechanical, and a hard violation when broken. Was it the right tool — a judgment call. Were the arguments right — usually checkable, and the most common silent failure. Did the call advance the task — the fuzziest, and the one that should never fail a gate by itself.
Whether a tool was allowed needs no model. Declare the allowlist and the forbidden set up front, and every call outside it is a violation, full stop. This is the cheapest, most reliable check available and it is routinely skipped in favour of harder ones.
Worth pairing with a declared risk classification for sensitive tools: what the consequence scope is, whether the action is reversible, whether it requires authorization, and whether it becomes dangerous in combination with another tool. Then cross-reference what was declared against what actually got invoked across the run set. That comparison — declared versus observed — is what a governance reviewer is really asking for.
Picking the right tool and then calling it wrong produces a plausible result that answers a different question than the one asked. A date range off by a year, an ID from the wrong entity, a filter that excludes the relevant rows — the call succeeds, returns data, and the agent proceeds confidently.
Much of this is checkable without a model, against the tool's schema and against values that appear in the task itself. The rest needs the same treatment as any other claim: does the evidence in the run actually support the argument the agent chose?
Repeated tool calls all look like waste in aggregate, and treating them as one metric hides what's actually wrong:
Each has a different fix. A single "redundant calls" number tells you none of that.
"Was this call worth making" is a much fuzzier judgment than "does this claim match the evidence," and it deserves a gentler severity. An agent that explores a dead end and recovers may be behaving perfectly well.
Report it, keep it visible, and let it inform a human — but a fuzzy judgment failing a release gate on its own is how teams learn to ignore the evaluator.
The short answer above is ours, and we've written at length about how we got to it: