Agent debugging must preserve rejected alternatives, not only chosen actions

Agent debugging must preserve rejected alternatives, not only chosen actions. A trace that says the agent called tool B after reading document X records the outcome of a decision, not the decision boundary. When B was wrong, the useful question is often why tool A, clarification, escalation, or no action lost. Without that candidate set, debugging becomes storytelling after the fact.

Conventional distributed traces follow the executed path because services are deterministic enough that the path usually reveals the fault. Agents insert a probabilistic selection step between observation and action. The same context can produce several plausible tools, plans, or answers. A waterfall can show latency, errors, and propagation across the chosen chain while hiding that the correct option ranked second by a fraction or disappeared because its description was truncated.

I do not mean storing private chain-of-thought. Production systems need a structured decision record, not unrestricted internal monologue: candidate identifiers, normalized scores or ranks where available, policy filters, validation failures, budget constraints, and concise declared reasons. Record which tools were eligible, which were removed by permissions, which arguments failed schema validation, and whether clarification was considered. This is enough to locate the failing layer without pretending that prose reasoning is a faithful causal explanation.

Rejected alternatives distinguish different remedies. If the right tool was never retrieved, fix discovery or context assembly. If it was retrieved but ranked low, inspect descriptions and selection evaluation. If it ranked first but policy denied it, examine authority rather than prompting. If the agent selected it and validation rejected malformed arguments, improve the schema or constrained generation. The visible failure may be identical in every case; the engineering response is not.

Counterfactual replay then becomes possible. Restore the context snapshot, substitute a candidate tool, router, policy, or model, and compare the downstream result. The goal is not exact reproduction of every token. It is testing a causal hypothesis against preserved state: would the old run have succeeded under the proposed change? Trajectory replay becomes much more useful when it can branch at the decision boundary instead of merely reenacting the chosen path.

The same record strengthens evaluation. Offline benchmarks measure whether the final task completed, but production learning needs near misses. A correct result chosen with unstable margins is a regression waiting for a small context shift. A wrong result where the correct alternative consistently ranks second suggests a different intervention from a run where it never appears. Aggregate those patterns and tool-selection tests can gate releases before completion rates visibly fall.

There is one precise concession: preserving every candidate at every token or planning step is expensive, noisy, and potentially sensitive. High-volume, low-consequence paths should sample or retain only bounded decision summaries, and raw context should follow strict access and expiry rules. The claim is not “log everything.” It is that consequential decision points need enough structured alternatives to test why the chosen edge won.

Telemetry should preserve decisions because an action log answers only what happened. Rejected alternatives reveal the shape of the choice the system faced. That shape is where retrieval, policy, ranking, validation, and budget interact — and where most agent failures are actually designed. It also makes disagreements inspectable instead of reducing them to a single opaque winner. If the evidence disappears after selection, the team can observe the incident but cannot reliably improve the system that produced it.