Route through a gateway you control; do not standardise on a vendor's
Every client pattern goes through a gateway the delivery team owns the config for. Vendor gateways are fine as a backend, never as the routing policy.
Tier is not strength
Tier says how much we know. Strength says how hard we are telling you to act. Scored independently.
Machine-readable target
{
"configKey": "gateway.routing.default"
}Nothing consumes it yet. Day two: findings ship as defaults into the gateway, routing config and skill library.
Body
x-gateway-routing ran cost-aware routing across three model gardens (Anthropic, Google, and an AU-region open-weight host) on six live task types for eight weeks. Routing by task type cut blended cost 38% at equal or better eval scores on four of the six tasks; the two agentic-loop tasks were the exception and stayed on frontier (c-ai-gateway-1, c-ai-gateway-2). None of that was possible when the routing policy lived in a vendor's product, which is what killed g-single-gateway-vendor.
The specific failure in the graveyard: one commercial gateway, chosen for its console, silently pinned two clients to a deprecated model version for eleven days when the vendor rolled a default. Nobody on our side could see the routing table. That is the entire case for owning the config. It is a text file, versioned, reviewed, and it is where standing answers about model choice get written to (c-ai-gateway-4).
The gateway you control can be a thin open-source proxy in front of vendor gateways. The recommendation is about who holds the routing policy and the fallback list, not about writing an inference server. Fallback order, per-task model pins, spend caps and the audit log are the four things that have to be yours.
Where this bites: government and banking engagements where the model list is a compliance artefact. A routing table under our change control satisfied one APRA CPS 230 material-service-provider review in July; a vendor console did not.
What it rests on
Static task-class routing rules capture most of the achievable cost saving (30–35%); learned routers add little and lose outside their training distribution.
Cost attribution per use case is the feature clients buy the gateway for; routing is secondary.
A commercial gateway is cheaper over three years than an owned open-source one once staffing is included.
Field
AI GatewayExperiment · validated
Cost-aware routing across three model gardensGraveyard · superseded
One commercial gateway over every model garden “Standardised on someone else's roadmap.”