Counterintuitively, a much LONGER system prompt can drive down cost per call below a shorter one. Explain when this flips, and derive the rough hits per write threshold above which the verbose but cached prompt wins. Be concrete about the discount mechanism on at least one real provider (Anthropic or OpenAI).
A long cached system prompt costs more on the first call (the cache write) and less on each later hit (about 0.1x input price). It wins only if its read cost is below a short prompt's full cost.
Think of a gym with two memberships. The cheap one charges five dollars every time you walk in. The expensive one charges fifty dollars to sign up, then fifty cents per visit. If you visit once, the cheap one wins. If you visit a hundred times, the expensive one wins by a lot. Long cached prompts are the expensive membership: an extra charge on the first call, then a small charge per call after. Once you visit (call) enough times, it pays off. It pays off only if the small charge per visit is lower than the cheap plan's price per visit, and it pays off sooner when that charge is very small.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Prompt caching breaks the old cost rule that shorter prompts are always cheaper. With caching, a long, information-dense system prompt can cost less per call than a short one, as long as the call pattern produces enough hits to amortize the write and the cached read cost stays below the short prompt's full cost.
The per-token price comparison still looks unfavorable: a 5000-token prompt has ten times more tokens than a 500-token one, and the list price per million tokens does not change. The difference is in the per-call cost after amortization. The first call pays a write premium that makes the verbose option more expensive. Every later call within the TTL pays the read price, often 0.1x of the input price, so the verbose option can match or beat the short option per call.
The break-even formula applies to any workload, and it sorts workloads into three regimes by where they sit relative to that break-even. For prompt design, this means a longer prompt that improves quality no longer has to cost more, as long as it is cached and the numbers work out.
The provider pricing model
Major providers offer prefix caching with two input prices. The first call that puts a prefix in the cache pays the write price; every later call that matches the same prefix exactly pays the read price.
Anthropic prices the write at 1.25x the normal input rate for the default 5-minute TTL (2x for the 1-hour TTL), and the read at 0.1x on most models (a 90% discount; 0.05x on Claude Opus 5.5 and Sonnet 5.5). The premium and discount apply only to the cached portion of input; anything past the cache breakpoint is billed at the normal input rate. Each hit refreshes the cache lifetime at no extra cost.
OpenAI caching is automatic. On GPT-5.6 and later, writes cost 1.25x and reads 0.1x on most models, the minimum cacheable prompt is 1,024 tokens, and entries last at least 30 minutes after the latest write or reuse. On GPT-5.5 and earlier there is no write charge and the cached rate depends on the model (0.1x on GPT-5.5).
Other providers also discount cached input, with their own rates and rules.
So the per-call cost of a long cached prompt depends on three numbers: the prompt length L, the write multiplier w and the read multiplier r. They replace the single price per token used in cost estimates without caching. With those three inputs, the amortization analysis is mechanical.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Anthropic's prompt-caching launch in 2024 reported up to 90% lower cost on long prompts; with 1.25x writes and 0.1x reads on most models, caching pays off after one cache read with the 5-minute TTL.
- OpenAI prompt caching is on by default; GPT-5.5 bills cached input at 0.1x with no write charge, so caching the same prompt costs less from the second call on.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat happens to the break-even formula when output tokens are factored in?
Output cost adds (N+1)Op_o to both sides equally, so it cancels in the break-even derivation. The output cost determines the absolute bill size, but not where the crossover sits. The crossover is purely a function of input-side caching arithmetic.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating prompt-caching value as a flat discount on every call. The actual structure has a write premium on the first call and a read discount on subsequent hits, and the break-even is amortized over the hit count, not per call.
60 second bullets to scan on the way to the call.
Explain how prefix caching prices write versus read tokens on major providers.
Recite the cache-cost formula across K calls.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.