Prompt Caching Economics: How the 90% Discount Really Works

Prompt caching has a reputation as free money. Turn it on, watch input costs drop by up to 90%, move on. That reputation is roughly half true.

The half that's true: for high-traffic applications with a large shared prompt prefix, caching is the single biggest line-item discount available anywhere in the LLM pricing model. Nothing else comes close.

The half that's false is hiding in two details most teams never read. First, writing to the cache isn't free — on providers with explicit caching, a cache write costs more than sending the same tokens uncached. Second, the discount only materializes when subsequent requests actually hit the cache, and production systems are full of patterns that quietly prevent that. A team can have caching "enabled," pay the write premium on every request, and collect almost none of the read discount. Their bill goes up, not down, and nothing in the application looks broken.

We covered the common cache-busting mistakes in our token optimization field guide. This post goes one level deeper: how the pricing actually works, when caching pays for itself and when it doesn't, and what a healthy cache economy looks like in production numbers.

The Pricing Model, Read Closely

All major providers now offer some form of prompt caching, but the mechanics differ enough that the same application can have very different cache economics depending on where it runs. As of mid-2026, the landscape looks like this:

Three details in that table drive everything else in this post.

The minimum cacheable prefix. Providers require a minimum prefix length before anything is cached at all — typically around 1,024 tokens. A 600-token system prompt caches nothing on most models. This produces a genuinely counterintuitive interaction: if you aggressively minimize a system prompt below the threshold, you can lose a 90% discount on it. Prompt minimization and prompt caching are both good levers, but they trade off against each other near the threshold, and the math needs to be done per endpoint.

The write premium. On Anthropic and Bedrock, populating the cache costs a premium over base input price. That premium is the price of an option: it only pays off if the cache gets read before it expires. Which leads to the question almost nobody asks.

The TTL. Caches live for minutes, not hours (unless you pay a higher write premium for an extended lifetime). Traffic patterns matter as much as prompt structure. A prefix that's requested twice a day never hits cache no matter how perfectly it's structured.

The Break-Even Math

Take a provider with a 25% write premium and a 90% read discount, and a shared prefix of any size. Call the base input cost of that prefix p. If n requests share the prefix within the cache lifetime:

# Uncached:  n requests at full price
cost_uncached = n × p

# Cached:    1 write (1.25x) + (n−1) reads (0.1x)
cost_cached   = 1.25p + (n−1) × 0.1p

# Break-even: cost_cached < cost_uncached
# 1.25 + 0.1(n−1) < n   →   n > 1.28

So with a 25% write premium, caching pays for itself from the second request onward. That's a forgiving threshold, and it's why caching is usually the right call for anything interactive. The extended-TTL tier at a 2x write premium breaks even at n > 2.1 — you need three requests inside the window, which is a real constraint for lower-traffic agents and batch jobs.

The inverse also matters and gets no attention: a cache write that's never read costs 25% more than not caching at all. An endpoint with a hit rate near zero — because its prefix churns per request, or its traffic is too sparse for the TTL — is paying a surcharge for nothing. We've seen endpoints where "enabling caching" was a net cost increase, sitting undetected for months because nobody was tracking cache economics per endpoint.

Hit Rate Decides Everything

Here's the same worked example at two different hit rates. A support copilot with a 3,000-token shared prefix (system prompt, tool definitions, policy context), 200,000 requests a day, base input at $3 per million tokens.

# Prefix spend, uncached baseline
200K requests × 3K tokens × $3/M            = $1,800/day

# At an 85% hit rate (25% write premium, 90% read discount)
hits:   170K × 3K × $0.30/M                 = $153/day
misses:  30K × 3K × $3.75/M                 = $338/day
total                                       = $491/day   (73% below baseline)

# At a 30% hit rate — same config, same "caching enabled"
hits:    60K × 3K × $0.30/M                 = $54/day
misses: 140K × 3K × $3.75/M                 = $1,575/day
total                                       = $1,629/day  (10% below baseline)

Same application, same configuration checkbox, same provider. The difference between a 73% saving and a 10% saving is entirely in the hit rate — and the 30% scenario is not a hypothetical. It's what cache performance commonly looks like after a few months of unmonitored prompt changes.

What Kills Hit Rates in Production

The classic cache-busters — timestamps in the system prompt, shuffled few-shot examples, personalization injected mid-prefix — are covered in the token optimization guide. In practice, the more expensive failures are organizational rather than mechanical, because they arrive silently through normal engineering work:

• Prompt A/B tests. Split traffic 50/50 across two prompt variants and each variant caches separately. Hit rates drop for both arms for the duration of the test. Sometimes that's a price worth paying; it should at least be a known price, factored into the test's cost.
• Deploys that touch tool definitions. Tool and function schemas usually sit inside the cached prefix. A deploy that adds a parameter, reorders tools, or regenerates schemas from code invalidates the entire cache at once. The bill spikes for an hour after every release, and on a chart it looks like traffic.
• Per-tenant preambles placed too early. Multi-tenant products often inject tenant configuration near the top of the prompt. Everything after that point becomes per-tenant cache, so a 5,000-tenant product gets 5,000 small caches instead of one hot one. Restructuring so shared content precedes tenant content routinely doubles hit rates.
• Sliding-window conversation history. In multi-turn applications, truncating old turns from the front of the history shifts every byte of the prefix and misses cache on every turn. Append-only history (with summarization instead of truncation when context gets long) keeps each turn's prefix a superset of the last, which is exactly what prefix caching wants.
• Traffic split across regions or accounts. Caches are scoped per provider account and often per region. A load balancer that alternates between two Bedrock regions halves the effective request rate seen by each cache — sometimes pushing sparse-traffic endpoints below the TTL threshold entirely.

None of these show up as errors. Latency is fine, responses are correct, dashboards are green. The only signal is a cache hit rate that drifted from 85% to 40% over a quarter, and the only place that signal exists is in per-request usage data most teams never look at.

Structuring Prompts for Cacheability

The design rule is simple to state: order prompt content from most static to most dynamic, and keep the boundary honest.

1. System instructions        — static across all traffic
2. Tool / function definitions — static between deploys
3. Few-shot examples           — static, fixed order
4. Tenant or feature context   — semi-static, per tenant
5. Conversation history        — append-only
6. Current user input          — fully dynamic, never cached

Everything above the first dynamic byte is cacheable; everything below it is not. The audit, then, is finding content that sits higher in the prompt than its volatility justifies. A timestamp in layer 1 poisons layers 1 through 5. Tenant config in layer 2 fragments the cache across your customer base. On providers with explicit cache breakpoints, place one at each boundary where the layers above it are stable — that way a tenant-level change invalidates only the tenant layer, not the tool definitions above it.

One more design note for agentic workloads: agents are unusually cache-friendly, because each step in an agent loop re-sends the same system prompt, the same tool definitions, and a growing (append-only) action history. A well-structured agent can see hit rates above 90% within a session. A poorly structured one — regenerating tool schemas per step, or re-serializing history in a different order — pays full price for some of the heaviest prompts in production.

What Good Looks Like

Cache hit rate benchmarks vary by workload shape, so a single global target is misleading. Rough reference points from production generative AI systems:

Beyond the hit rate itself, two derived numbers are worth tracking per endpoint. The write-to-read ratio tells you whether you're paying premiums for caches that get used (a ratio drifting toward 1:1 means you're writing caches nobody reads). The effective input price — blended $/M tokens actually paid across cached and uncached input — is the single number that captures whether your caching strategy is working, and it's the number to alert on after deploys.

Making Cache Economics Visible

Every number in this post comes from data your providers already expose: per-request token counts split by cached and uncached input. The hard part isn't access, it's assembly — joining that usage data across Bedrock, OpenAI, and Anthropic, attributing it to the feature and team that generated it, and watching the trend per endpoint so a hit-rate regression shows up as an alert instead of a quarterly billing surprise.

That assembly work is what MLCostIntel's generative AI cost tracking does. Cache hit rates by endpoint, effective input price by feature, write/read ratios, and anomaly detection when a deploy quietly resets your caches. If your team enabled prompt caching a while ago and nobody has verified what it's actually saving since, that's exactly the kind of question a free assessment answers.