Supervision trees contain failure better than global retry loops

I trust a supervision tree more than a global retry loop because it makes failure containment part of the topology. A retry loop asks only whether the same operation should run again. A supervisor asks the harder questions first: what failed, which state is now suspect, which dependants share that state, and where should the failure go if local recovery stops making progress? For an agent runtime, that difference determines whether one bad tool call becomes a repaired worker or a fleet-wide cost incident.

Global retry is attractive because it is easy to add at the orchestration boundary. Catch an exception, back off, and replay the run. But an agent run is not one operation. It is a graph of model calls, tools, memories, queues, credentials, and side effects. Replaying the whole graph treats healthy work as collateral damage. It can repeat a payment, discard valid research, rebuild expensive context, and amplify pressure on the same dependency that failed. The policy has no structural knowledge, so its safest-looking response often has the largest blast radius.

A supervision tree encodes that missing knowledge. Workers perform the task; supervisors own recovery policy for defined groups of workers. I would place independent fan-out agents under a one-for-one policy, so a failed verifier restarts without disturbing the other verifiers. I would treat a tightly coupled debate or consensus round as one unit when its participants share assumptions that cannot survive independently. For a sequential pipeline, I would preserve valid upstream results while invalidating the failed stage and everything downstream that depends on it. Recovery scope then follows dependency scope instead of process scope.

This is more than a neat translation of Erlang/OTP into agent language. It forces me to draw the dependency map that a retry wrapper lets me avoid. If a session handler restarts while its cache retains corrupted state, the restart is cosmetic. If a planner fails after emitting an invalid plan, restarting only an executor leaves the cause untouched. The tree makes these relationships reviewable: siblings are grouped because they are independent or coupled for a reason, and escalation boundaries document which state can be trusted after each class of failure.

Akka adds another useful discipline: recovery can be selected by failure type rather than hidden inside business logic. A transient provider timeout may justify restart; a malformed tool protocol may require stop; a violated invariant should bubble to a parent with authority to quarantine the larger unit. I do not want every agent prompt or tool adapter improvising those decisions. The runtime should classify failure, apply a declared strategy, and record the transition. That keeps the recovery policy deterministic even when the agent behaviour is not.

The most important supervisor decision is to stop retrying. Restart intensity turns persistence into a budget: tolerate a bounded burst within a time window, then terminate the local subtree and escalate. Without that ceiling, self-healing becomes a restart storm. A bad prompt, impossible task, or permanently misconfigured tool can consume tokens and initialization time forever while dashboards celebrate successful restarts. Hierarchies also require careful budgets, because generous limits at several levels multiply. I want tighter limits as failures climb, plus a circuit breaker, spend cap, and incident signal when a subtree exhausts its allowance.

This does not eliminate the need for retries. If a call is idempotent, the failure is demonstrably transient, and no dependant state can have changed, a bounded local retry with backoff and jitter is simpler than restarting a worker. That is the boundary I use: retries belong inside the smallest verified fault boundary, not around the whole agent run. Once partial side effects, shared state, or downstream dependencies exist, the supervisor needs enough context to choose restart, stop, or escalation.

I therefore design the failure tree alongside the execution graph. Every node gets an owner, restart semantics, an idempotency story, an intensity budget, and an escalation target. Observability follows the same shape: I want to see which subtree failed, what was restarted, what state was invalidated, and why escalation occurred. Global retries optimize for getting back to green. Supervision trees optimize for preserving everything that is still correct while containing what is not. In production agent systems, that is the more valuable definition of recovery.