O&A
O&A Consulting · Answer

How do you evaluate an AI agent from its execution trace?

In four passes. Normalize — OTel GenAI spans, Bedrock traces, plain JSONL all map into one canonical shape, or every check downstream has to know about every framework. Run deterministic checks first, because much of what matters is mechanical. Then a model, only for what genuinely needs language understanding, and only to extract findings with pointers into steps. Then score in ordinary code, so the same trace gives the same verdict tomorrow without a model in the loop.

Trace AnalysisOpenTelemetryAgent EvaluationObservability
01 Answer

What the trace has to contain

Evaluation quality is capped by capture quality. Before anything else, check that the export actually carries:

  • Tool calls with their arguments and results. A span that records "called search" without what was asked or what came back supports almost no checking.
  • The sources the agent retrieved — the passages themselves, since a span saying retrieval happened leaves groundedness unanswerable.
  • Step ordering and identity, so a finding can point at a specific step and a reader can go look.
  • Errors, retries, and escalations, including the ones the agent recovered from silently.
  • Cost, tokens, and timing per step.

Where a field is missing, the honest output is "unverifiable" — which doubles as a useful signal about the instrumentation, rather than a failure of the agent.

02 Answer

Deterministic first, model second

The ordering is a cost and reliability decision. A surprising share of what a reviewer cares about needs no model at all, and mechanical checks are free, instant, and identical on every run:

  • Forbidden tool calls and allowlist violations
  • Budget ceilings on cost, duration, and call count
  • Output schema conformance
  • PII appearing where it shouldn't
  • Loops — the same call repeated consecutively
  • Retrying a call already known to have failed, unchanged
  • Escalating to a human and never getting an answer
  • Errors that were never recovered from

Only after those does a model earn its place, on questions like whether a claim is supported by a retrieved passage.

03 Answer

Long traces need navigation more than context

The naive approach stuffs the whole trace into one call, which silently truncates evidence the moment a run gets big — and agent runs get big. Raising the context limit postpones the problem rather than solving it.

The better shape is to give the model a closed set of read-only navigation tools over the trace and let it pull the slice each claim needs. Log every retrieval, and the investigation becomes replayable rather than an opinion — you can see exactly what the model looked at before it concluded anything.

04 Answer

Label what each finding rests on

The last pass matters more than it sounds. A finding backed by a retrieved source document and a finding backed by the agent's own account of a tool result are not equally strong, and collapsing them is how an evaluator starts making accusations it can't support.

Mark each one: independently observable ground truth, agent self-report, or nothing in the trace speaks to it. Let only the first fail a gate on its own. The rest still get recorded and shown.

05 Further

Where this is argued properly

The short answer above is ours, and we've written at length about how we got to it:

The answer was right. The agent made it up. ↗Why isn't output accuracy enough? →agent-trustkit — OTel and Bedrock traces in, evidence out →