Click any words you think contain an error. Click again to unmark.
Mark at least one word to submit.
**LLM APIs charge for two things separately.** Input tokens are what you send in, system prompt plus user message. Output tokens are what the model writes back. They have different prices, and the output price is usually three to five times the input price.
Prompt caching only discounts the input side. When you turn on caching, the provider stores the work it did for the leading bytes of your prompt and gives you a deep discount the next time you send exactly those same bytes. The discount is on input. Output is not cached, because every output is unique. Each generated token costs the full output rate.
The app in the question sends about 2200 input tokens and generates 3000 output tokens. If caching saves 90 percent on the cached portion of input, that is a real saving but a small slice of the total bill. Output is 3000 tokens at roughly four times the input price, which dwarfs even uncached input. So the realized bill reduction is somewhere around 15 percent, not 90 percent.
The rule is simple. Read the price sheet. The cache discount is a per-token input discount; it is not a per-invoice discount. Apply it only to the cached input tokens and recompute the total.
**Two errors compound.** First, the cache discount applies to input tokens only; output is billed at full rate every time. Second, output is typically priced at three to five times the input rate, so output dominates the invoice on any generation-heavy workload.
Walk the arithmetic on the scenario. Input is 2200 tokens. Of those, 2000 hit the cache at the 90% discount, equating to 200 paid input-tokens, plus the 200 user tokens at full input price, totaling 400 input-token equivalents. Output is 3000 tokens at full output rate. Assuming output is 4x input, output is worth 12,000 input-token equivalents on the bill. Total bill equivalents: 12,400.
Without caching, the same call costs 2200 input-equivalents plus 12,000 output-equivalents = 14,200. So the realized saving is roughly 1800/14,200 = 12-15 percent. The headline 90% figure overshoots reality by a factor of six.
The correct mental model has three terms: (cached_input * discount * input_price) + (uncached_input * input_price) + (output_tokens * output_price). The cache discount only multiplies the first term. On output-heavy workloads (writing assistants, code generation, chat completion) output dominates, and caching delivers single-digit to low double digit percent savings. On input-heavy workloads (long-context RAG with short responses, agent loops with massive tool schemas and brief tool outputs) the input share of the invoice is large and the saving approaches but never reaches the headline.
The ceiling on bill reduction from caching alone is the input share of the bill. If output is fifty percent of the bill, the maximum possible saving is fifty percent times the cache hit rate times the discount factor, even on a perfect workload. Always reprice on the actual workload profile before promising the headline.
Quick reference
TL;DR
Prompt caching discounts INPUT tokens only, not output.
Imagine a bakery that gives you ninety percent off flour but full price on the cake itself. If your order is mostly cake with a little flour, the discount barely shows up on the receipt. Prompt caching is the flour discount. It cuts the cost of the long, repeated instructions you send in, but it does nothing for the long answer the model writes back. When the model writes a three thousand word essay, that essay is the cake, and you pay full price for every word. The total bill drops a little, not a lot.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Input-heavy workloads with short outputs (long-context RAG, classification, extraction) When output is small relative to input, the cache discount approaches the headline figure on the bill, so caching really is the dominant cost lever and the math in this question understates its value.
Agent loops with massive stable tool schemas and brief tool outputs Tool schemas can run thousands of tokens, stable across turns; if outputs are short JSON, caching cuts the bulk of the bill in line with the headline figure.
Workloads where the cache TTL is consistently violated by low call rates If the cache expires before the next hit, the cache-write premium can exceed the realized savings and caching may slightly increase the bill. The math here assumes effective hit rates.
Provider tiers where caching is unavailable or carries no discount Some smaller or self-hosted setups do not offer the feature; the entire analysis is moot until the provider exposes a cache-pricing tier.
Quick scan · 2–4 min
Everything important, quickly.
3 min: name caching as input-only, build the bill formula, plug in the scenario numbers to derive ~12-15%, and explain the input-share ceiling on caching savings.
Real-world examples
Real products, models, and research that use this idea.
1Anthropic Claude pricing in 2026: input $3/M, output $15/M (5x), cache reads at ~$0.30/M (10x discount on input). A 90% input-token cut buys 12-15% bill cut on output-heavy chat.
2OpenAI o4 and GPT-5 pricing show output at 4x input with automatic prompt caching at ~50% discount; realized bill savings on generation-heavy apps land near 5-10%.
3Anthropic case studies pitch caching wins of 30-50% on workloads with long stable context and short responses (RAG over documents, repeated tool schemas), confirming the input share of bill ceiling.
4DeepSeek-V3 disaggregated pricing shows cache hits at 10% of input price and output at 4-5x input, the same arithmetic recovers.
5Cursor and Anthropic's Computer Use Tool both publish telemetry showing that long stable system prompts plus short tool calls is where caching delivers 30%+ bill cuts in production.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat is the maximum possible bill reduction from prompt caching alone, even with a perfect hit rate?
Sign in or go Pro to see the approach for this follow-up.
QWhy is output priced higher than input on most provider price sheets?
Sign in or go Pro to see the approach for this follow-up.
QHow would you redesign an output-heavy workload to recover more value from prompt caching?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Multiplying a per-token input discount by the whole invoice and predicting a 90% bill cut. Output tokens carry the bulk of cost on generation-heavy workloads and the cache does not touch them.