Counterintuitively, a much LONGER system prompt can drive down cost per call below a shorter one. Explain when this flips, and derive the rough hits per write threshold above which the verbose but cached prompt wins. Be concrete about the discount mechanism on at least one real provider (Anthropic or OpenAI).
A long system prompt costs more on the first call (the cache write) and cheaper on every subsequent hit (the cache read at ~10% on Anthropic, ~50% on OpenAI).
Think of a gym with two memberships. The cheap one charges five dollars every time you walk in. The expensive one charges fifty dollars to sign up, then fifty cents per visit. If you visit once, the cheap one wins. If you visit a hundred times, the expensive one wins by a mile. Long cached prompts are the expensive membership: an upfront premium on the first call, then a tiny per-call charge after. Once you visit (call) enough times, it pays off, and that crossover usually comes after just a few calls because the per-hit discount is so steep.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Prompt caching has inverted one of the oldest cost rules of LLM engineering: that shorter prompts are cheaper. They no longer are, at the workloads most production teams actually run. A long, information-dense system prompt that would have been a cost liability before caching is now a cost optimization, as long as the call pattern produces enough hits to amortize the write.
This flip surprises people because the per-token price comparison still looks unfavorable: a 5000-token prompt has ten times more tokens than a 500-token one, and the headline 'price per million tokens' figure is unchanged by caching. The flip happens in the realized per-call cost after amortization. The first call pays a write premium that makes the verbose option more expensive. Every subsequent call within the TTL window pays the read price, which is so deeply discounted that the verbose option matches or beats the short option per call.
The goal of this walkthrough is to give you the break-even formula in a form you can apply cold to any workload, and to classify workloads into three regimes by where they sit relative to that break-even. The arithmetic is small but the implications for prompt design are large: it means quality-driven prompt length is no longer a tax on cost, and shorter-is-cheaper as a default heuristic has been retired by the provider pricing structure.
The provider pricing model in 2026
Every major provider in 2026 offers prefix caching with a two-tier price structure on the input side. The first call that lands a particular prefix in the cache pays the write tier; every subsequent call that matches the same prefix bytes-exact pays the read tier.
Anthropic in 2026 prices the write at roughly 1.25x the normal input rate (a 25% premium for the storage), and the read at roughly 0.1x the normal input rate (a 90% discount). The premium and discount apply only to the cached portion of input; anything past the cache breakpoint is billed at the normal input rate. There are two TTL tiers: default 5 minutes and extended 1 hour, with the latter carrying a higher write premium.
OpenAI in 2026 prices the write at the normal input rate (no premium) and the read at roughly 0.5x the normal input rate (a 50% discount). The cache is automatic on the leading 1024+ tokens of the request, with TTL behavior varying by load.
Google Gemini and other providers have similar two-tier structures with provider-specific multipliers. The exact numbers move with releases, but the structure (write premium plus read discount) is the universal pattern.
The consequence for any cost analysis is that the per-call cost of a long cached prompt depends on three things: the prompt length L, the write multiplier w, and the read multiplier r. Together they replace the single 'price per token' that ungated cost reasoning depends on. Once you accept those three numbers as inputs, the amortization analysis follows mechanically.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Anthropic's prompt-caching launch in 2024 reported up to 90% cost reduction on long-context workloads (RAG, coding agents) with break-even at 2-3 cache hits.
- OpenAI's automatic prompt caching (2024+, GPT-4o and later) discounts ~50% on cached prefix tokens with no write premium, making the verbose-prompt win immediate from the second call.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat happens to the break-even formula when output tokens are factored in?
Output cost adds (N+1)Op_o to both sides equally, so it cancels in the break-even derivation. The output cost determines the absolute bill size, but not where the crossover sits. The crossover is purely a function of input-side caching arithmetic.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating prompt-caching value as a flat discount on every call. The actual structure has a write premium on the first call and a read discount on subsequent hits, and the break-even is amortized over the hit count, not per call.
60 second bullets to scan on the way to the call.
Explain how prefix caching prices write versus read tokens on major providers.
Recite the cache-cost formula across K calls.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.