Parallel tool calls need effect isolation before they need concurrency

Parallel tool calls need effect isolation before they need concurrency. Fan-out can cut a turn’s latency to the slowest branch instead of the sum of every branch, but that speedup is safe only when calls do not race over shared state, duplicate irreversible work, or conceal a partial commit behind one failed batch.

The usual implementation makes concurrency look mechanical: launch several calls, gather their results, and return them together. That works for independent reads. It becomes a transaction problem when one call reserves inventory, another charges a card, and a third sends confirmation. If the confirmation fails after the first two succeed, retrying the whole batch can charge twice. Declaring the batch failed erases two real effects. The runtime needs a state model, not a faster event loop.

I start by classifying every call as read-only, idempotent write, compensatable write, or irreversible effect. Then I record its dependencies, resource scope, idempotency key, timeout, and commit status. Only calls whose effect sets are disjoint — or whose conflicts are controlled by the underlying system — are eligible to run together. Model-produced independence is a proposal; the orchestrator must verify it against declared tool contracts.

Partial failure should preserve branch truth. Successful calls return committed results; failed calls return typed errors; timed-out calls remain unknown until the system reconciles whether an effect happened. Tool errors must be typed for the agent so it can distinguish repairable arguments from transient failure and ambiguous completion. Retrying only the failed subset is safe only after the successful subset is durably recorded and every retry carries the original idempotency identity.

This is where idempotency becomes the cheapest reliability. It does not mean repeating any call is harmless. It means the receiving boundary recognizes the same logical operation and returns the original outcome instead of creating another effect. For non-idempotent systems, the orchestrator needs an outbox, reservation protocol, compensation path, or human reconciliation queue. Parallelism does not remove those obligations; it makes their absence arrive sooner.

Stragglers need the same discipline. A per-tool timeout can release the gather barrier, but cancellation is not proof that the remote operation stopped. The result should be “completion unknown,” not “failed,” until status is queried. Multi-agent systems need negotiated backpressure because unlimited fan-out shifts queue pressure and rate-limit failure into dependencies that cannot explain the global plan.

The gather step is itself a policy decision. Some tasks require every branch before proceeding; others can commit after a quorum, a deadline, or the first verified answer. That reduction rule must be explicit, because silently dropping a slow fraud check is different from omitting a redundant search result. Completion criteria belong to the workflow contract.

There is one precise concession: broad parallelism is a sensible default for side-effect-free reads over independent resources when stale or partial results are acceptable and each result carries its source and timestamp. Search, metadata lookup, and document fetches often fit. Even there, tail latency, rate limits, and inconsistent snapshots should be visible rather than flattened into one confident answer.

I want the trace to show fan-out group, branch dependencies, effect class, start and finish states, retries, cancellation attempts, and the reduction rule that combined results. That makes the effect the schedulable unit. Concurrency is then an optimization applied after correctness boundaries are known. The production question is not whether tools can execute at the same time. It is whether each effect remains attributable, recoverable, and non-duplicated when they do.