Rate limits are fairness policy for agent fleets, not API protection

Rate limits are fairness policy for agent fleets, not merely protection for an upstream API. A provider’s requests-per-minute ceiling tells me when its service will reject traffic. It does not tell me how a shared fleet should divide scarce tokens, concurrency, latency, or tool capacity among tenants whose agents produce radically different demand. That allocation is a product and governance decision, whether the platform names it or leaves it to queue arrival order.

First-come, first-served feels neutral because it does not inspect identity. In a fleet it rewards the noisiest producer. One tenant’s recursive planner, high fan-out workflow, or burst of long contexts can raise queue age for everyone else while remaining technically below the global provider limit. A single request is not a stable unit either: one call may consume a few hundred tokens and another tens of thousands. Fairness measured by request count therefore converts heterogeneous work into an advantage for expensive callers.

I would separate rate limiting from fair scheduling. Token buckets answer whether a tenant may inject more demand under a quota and burst allowance. Fair queues answer which admitted work should run next. Backpressure answers what happens when capacity is unavailable. Conflating these produces brittle behavior — a hard cap can prevent overload but still permit starvation, while fair ordering can protect quiet tenants without limiting total consumption by a loud one. The controls need different semantics because they solve different problems.

Weighted virtual time is a useful scheduling model. Each tenant advances according to the estimated cost of its work divided by its entitlement, and the scheduler favors the smallest virtual finish time. Idle tenants should not accumulate an unlimited advantage, and a maximum virtual-time gap should prevent pathological starvation after long imbalance. Managed queue mechanisms can implement a simpler version by using a tenant key to identify noisy and quiet groups, reordering work so one group does not inflate another’s dwell time. The important point is that identity becomes an allocation boundary, not merely an audit label.

Agents also consume multiple resources. Tokens, model concurrency, tool slots, memory, and execution time can each become the bottleneck. Dominant Resource Fairness offers the right intuition: compare tenants by the largest share they consume of any scarce resource, then allocate to reduce unequal dominant shares. Exact DRF may be too heavy for every runtime, but the principle prevents a token-light, tool-heavy workflow from evading policy because the platform meters only language-model calls.

Priority is part of fairness but cannot replace it. Interactive work may deserve lower latency than batch enrichment, and paid service tiers may have explicit weights. Pure priority queues, however, can starve background work indefinitely. Age-based promotion or guaranteed minimum shares keep lower-priority classes alive. The policy should be visible in metrics: per-tenant queue age, throttled demand, dominant share, admission delay, and starvation events. A global 429 rate hides every one of these distributional failures.

There is one precise concession: a small single-tenant deployment with homogeneous calls can use a global token bucket as ordinary API protection because there is no meaningful allocation dispute to resolve. Once workloads vary or tenants share the fleet, that simplicity becomes an undocumented policy of “whoever arrives fastest wins.” The architecture is already making a fairness decision — it is just making it badly.

I would therefore review fleet rate limits the way I review authorization rules. Who receives capacity, in what unit, with what burst, what minimum service, and what evidence when demand is deferred? Those answers define the experience of a shared agent platform. Provider limits are only an external constraint. Fairness policy is how the platform decides whose work survives inside it.