'Prompt caching cuts our bill by 90 percent across the board', what's overclaimed?

**LLM APIs charge for two things separately.** Input tokens are what you send in, system prompt plus user message. Output tokens are what the model writes back. They have different prices, and the output price is usually three to five times the input price. Prompt caching only discounts the input side. When you turn on caching, the provider stores the work it did for the leading bytes of your prompt and gives you a deep discount the next time you send exactly those same bytes. The discount is on input. Output is not cached, because every output is unique. Each generated token costs the full output rate. The app in the question sends about 2200 input tokens and generates 3000 output tokens. If caching saves 90 percent on the cached portion of input, that is a real saving but a small slice of the total bill. Output is 3000 tokens at roughly four times the input price, which dwarfs even uncached input. So the realized bill reduction is somewhere around 15 percent, not 90 percent. The rule is simple. Read the price sheet. The cache discount is a per-token input discount; it is not a per-invoice discount. Apply it only to the cached input tokens and recompute the total.

**Two errors compound.** First, the cache discount applies to input tokens only; output is billed at full rate every time. Second, output is typically priced at three to five times the input rate, so output dominates the invoice on any generation-heavy workload. Walk the arithmetic on the scenario. Input is 2200 tokens. Of those, 2000 hit the cache at the 90% discount, equating to 200 paid input-tokens, plus the 200 user tokens at full input price, totaling 400 input-token equivalents. Output is 3000 tokens at full output rate. Assuming output is 4x input, output is worth 12,000 input-token equivalents on the bill. Total bill equivalents: 12,400. Without caching, the same call costs 2200 input-equivalents plus 12,000 output-equivalents = 14,200. So the realized saving is roughly 1800/14,200 = 12-15 percent. The headline 90% figure overshoots reality by a factor of six. The correct mental model has three terms: (cached_input * discount * input_price) + (uncached_input * input_price) + (output_tokens * output_price). The cache discount only multiplies the first term. On output-heavy workloads (writing assistants, code generation, chat completion) output dominates, and caching delivers single-digit to low double digit percent savings. On input-heavy workloads (long-context RAG with short responses, agent loops with massive tool schemas and brief tool outputs) the input share of the invoice is large and the saving approaches but never reaches the headline. The ceiling on bill reduction from caching alone is the input share of the bill. If output is fifty percent of the bill, the maximum possible saving is fifty percent times the cache hit rate times the discount factor, even on a perfect workload. Always reprice on the actual workload profile before promising the headline.