cavendish
Do thisTestedstrength · moderate

Prompt caching: use for stable prefixes over 2k tokens; expect 30–45%, not 60%

Cache when the shared prefix is over ~2k tokens and reused inside the provider's TTL. Budget on 30–45% input-cost reduction across a real pattern; the 60% figure is a best-case single call.

Tier is not strength

Tier says how much we know. Strength says how hard we are telling you to act. Scored independently.

Evidence tierTested
Strengthmoderate
OwnerPRPriya Raman
Last validated28 Aug 2026
Review by26 Nov 2026
Half-life90 days
Citations61
VerticalsCross-sector
decay84d until review

Machine-readable target

{
  "configKey": "inference.prompt_cache.min_prefix_tokens"
}

Nothing consumes it yet. Day two: findings ship as defaults into the gateway, routing config and skill library.

Body

x-token-cost-ledger instrumented six client patterns for ten weeks and attributed every token to a cache hit, a cache miss or an uncached call. Across the six patterns, prompt caching reduced input spend by between 31% and 44% (c-token-cost-reduction-1). The single-call 60% number that vendors publish was reproduced on one pattern with a 14k-token system prompt and a tight loop, and on nothing else. That is the finding behind g-prompt-caching-universal.

The reasons the number is lower in practice are mechanical, not mysterious. Cache TTLs are short (five minutes on two of three providers), so anything with human latency in the loop misses. Prefixes drift because tool definitions change per call. And a good share of spend is output tokens, which caching does not touch at all (c-token-cost-reduction-2).

The threshold of ~2k tokens is where the cache-write premium is paid back inside two reuses on current pricing. Below it, caching is neutral to slightly negative. Above it, and with the prefix pinned, the saving is reliable. The ledger is the artefact that lets a team see which of their calls are actually hitting.

Strength is moderate rather than strong because pricing moves. Two providers changed cache-write pricing between June and August, and one shortened its TTL. This recommendation carries a 90-day half-life for that reason; the number itself lives in sa-cost-per-mtok, which refreshes faster. The 'not 60%' half is the part we are confident of (c-token-cost-reduction-4).

PRSigned Priya Raman · Research engineer · inference · 28 Aug 2026

What it rests on

Prompt caching yields 30–45% per-task savings on stable prefixes over 2k tokens and near nothing below; it is not a universal 60%.

Tested c-token-cost-reduction-1
86%

Cache hit rate is fragile to prompt ordering; a reorder that moves a dynamic field above the static prefix can invert the saving.

Tested c-token-cost-reduction-2
80%

Prompt caching is a universal 60% cost cut.

Assessed c-token-cost-reduction-4
12%

Field

Cost redux on tokens

Experiment · validated

Token cost ledger across six client patterns

Graveyard · refuted

Prompt caching as a universal 60% cost cut Sixty percent, on the vendor's workload.