Put backpressure on agent fan-out before you put it on the model
Bound the number of in-flight sub-agents and tool calls at the orchestrator with a queue that applies backpressure. Model-side rate limits are the wrong place to discover you have a fan-out problem.
Tier is not strength
Tier says how much we know. Strength says how hard we are telling you to act. Scored independently.
Machine-readable target
{
"taskType": "agent-fan-out",
"configKey": "agents.orchestrator.max_inflight"
}Nothing consumes it yet. Day two: findings ship as defaults into the gateway, routing config and skill library.
Body
x-backpressure-queue reproduced the failure pattern from two client incidents in a harness: an orchestrating agent spawning sub-agents on a per-item basis, each of which spawned tool calls, until the provider's rate limiter started rejecting and the retries made it worse. With a bounded in-flight queue at the orchestrator, the same workload finished 3.4× faster at a third of the token spend, because nothing was retried (c-backpressure-1, c-backpressure-2).
The reason to put the limit at the orchestrator rather than trusting the model provider's 429s: the provider limit is global to your account, so one runaway pattern degrades every other pattern sharing the key. In the second client incident, a nightly batch job took a live servicing agent down for forty minutes that way (c-backpressure-3).
What to set: a max in-flight per orchestrator, a per-tool concurrency cap, and a queue depth that fails fast rather than growing unbounded. The numbers are workload-specific; the shape is not. The reference implementation is a 200-line wrapper that has been dropped into three patterns since July.
Strong because the mechanism is not statistical. It is queueing theory and it behaves the same way every time.
What it rests on
Under load spikes, unbounded agent fan-out fails on provider rate limits before it fails on latency, and retries amplify spend two to four times.
Admission control at the orchestrator with a per-task cost cap bounds spend to about 1.1× and holds p95 within 2× under a 10× spike.
Provider rate limits are moving to token-based priority tiers, which makes client-side admission control more valuable, not less.
Field
BackpressureExperiment · voting
Backpressure patterns for agent fan-out