AI agent traceability: how to trace what an AI agent did.

Short answer

To trace what an AI agent did, record every step of the run as it happens: the tool or model call, the agent that made it, the exact inputs and observed outputs, and a timestamp. Link the steps in sequence so the order is reconstructible and a missing step shows up as a visible gap. If the record also carries a cryptographic commitment (a hash) over each step's bytes, the trace becomes independently verifiable: anyone can confirm the record was not altered without trusting the operator. That combination, a complete ordered trace plus per-step commitments, is what turns tracing from debugging into evidence.

Why tracing an agent is hard

A traditional program does the same thing given the same input. An agent does not. The same prompt can produce different tool-call sequences on different runs, different models, or different days. A run fans out across tools, sub-agents, and retries. Some steps happen in parallel. Some calls fail and get retried with different arguments. By the time a human asks "what did it actually do," the only thing left is usually a summary written by the same agent, which is a story, not a trace.

This is why traceability has to be built in at capture time. Reconstructing a run from fragments, logs, and chat transcripts is archaeology. Capturing each step as it executes is instrumentation. The teams that can answer "what did the agent do" in an audit are the ones that instrumented, not the ones that kept good notes.

Traces show the path. Receipts prove the path is real.

There are two levels of traceability, and they get conflated constantly. The first level is a trace: an ordered record of the steps, built for the operator. It tells you what happened so you can debug it. The second level is a verifiable trace: the same ordered record, but every step carries a cryptographic commitment anyone can recompute, so a skeptic with no access to your systems can confirm nothing was edited, reordered, or deleted.

The difference matters when the trace leaves your building. A client dispute, an audit, a regulator, an insurance claim: in every one of these, the trace is offered by the party with an interest in what it says. A plain trace asks the reader to trust the logger. A verifiable trace removes trust from the equation: recompute the hashes, confirm the sequence, and the trace checks out regardless of who produced it.

What to capture per step

A traceable step needs enough fields that a stranger can follow the run and check each entry. The minimum set:

• A stable step identifier. A receipt ID or step ID that never changes and can be referenced later.
• Agent identity. Which agent (or sub-agent) performed the step. Multi-agent runs without per-step identity are untraceable by construction.
• The tool or model called. The exact operation, not a paraphrase. Tool name, endpoint, model version.
• Exact inputs in canonical bytes. What went in, byte for byte, in a canonical encoding so the hash is deterministic.
• Observed outputs in canonical bytes. What came back, byte for byte. Observed, not summarized.
• Execution-time timestamp. When the step ran, captured at execution, not when the log was written.
• A SHA-256 commitment over the step bytes. What makes the step independently checkable.
• Sequence position. Where the step sits in the run's order, so a missing step is a visible gap.

This field set is exactly what the open AER-1 format (draft-zambo-aer1, an IETF Internet-Draft) defines for a verifiable execution receipt: one receipt per action, with canonical bytes and verification rules anyone can implement.

The capture point is everything

The same record captured in two different places has completely different evidentiary value. A trace reconstructed from logs after the run is a summary with extra steps: it reflects what the system chose to remember, in the form it chose to present. A trace captured at execution, inside the tool-call path itself, reflects what actually ran.

Practically, this means hooking the framework's own execution path: callbacks, tracing processors, event buses, middleware. The step record is emitted when the tool executes, before the agent summarizes it, before anything downstream can rewrite it. If the agent summarizes the run afterward, the summary can be checked against the captured steps, which is exactly how you catch a summary that drifts from what actually ran.

Collectors that do this already exist for the major agent frameworks. One install, one hook, and every run emits a verifiable receipt per step. See the framework collectors on the hub index.

Sequence gaps: the part most traces miss

A trace with steps 1, 2, 3, 4 tells a story. A trace with steps 1, 2, 4 tells a different story: step 3 is missing. But most agent traces never number their steps, so a missing step is invisible. The trace simply shows what is there, and nobody knows what is not.

Traceability worth the name needs gap detection: every step carries its position in the run's sequence, so a verifier can confirm the chain is complete. A one-step receipt cannot demonstrate that the run had only one step. A sequenced receipt set can, because the verifier checks that the positions form an unbroken chain. When someone asks "did the agent do anything it is not showing you," sequence position is the mechanism that answers.

What traceability does not do

A complete, verifiable trace proves the recorded actions happened as recorded and were not altered afterward. It does not prove the agent was right, the policy was wise, or the outputs were correct. It does not prove the upstream capture was truthful: if the tool itself lied about its result, the receipt faithfully records the lie. Traceability establishes what happened. Whether what happened was good, safe, or correct is a separate layer of judgment.