Claude Sonnet (illustrative rates for this calculation): - input tokens: $3 per 1M - output tokens: $15 per 1M - cached input tokens: $0.30 per 1M A single API call has: - 10,000 cached input tokens (system prompt + tool schemas, cache HIT) - 2,000 new input tokens (this query's retrieved chunks + user question) - 600 output tokens (the assistant's reply) Compute the total cost of this single call in US cents (¢). Round to two decimal places. Then state which component (input / cached / output) dominates the cost.
Cost is a three-line sum: cached input plus new input plus output. Output tokens dominate because the output rate is 5x input and 50x cached input, even with far fewer tokens.
Imagine a print shop with three price tiers. Photocopying a page you already printed last week is almost free. Printing a fresh page costs a bit more. But writing a brand-new page by hand, word by word, is the expensive part. An API call works the same way. The huge system prompt you reuse every time is the cheap photocopy. This query's fresh context is the mid-price print. The model's reply is the hand-written page, generated one token at a time, billed at the top rate. So even a short reply can cost more than a giant cached prompt. To spend less, you do not trim the prompt first. You make the model say less.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Per-call cost is the most practical math in inference optimization, and it is where engineering meets the invoice. Every product decision, from how long a system prompt to ship to how chatty an agent should be, eventually shows up as a number on a billing dashboard. The skill an interviewer is probing is whether you can decompose that number into its parts and reason about which part to attack.
The trap built into this question is volume bias. The call has 10,000 cached tokens, 2,000 fresh input tokens, and only 600 output tokens. The instinct is to assume the 10,000-token bucket dominates because it is by far the largest count. It does not. Pricing is not uniform across token types, and the smallest bucket here carries the highest rate by a wide margin.
This deep dive walks the arithmetic line by line, ranks the components, connects the dollar cost to the underlying prefill versus decode mechanics that justify the rates, and then derives the correct cost-optimization order. By the end you should be able to do this math on a whiteboard in under a minute and explain, from first principles, why shortening the model's output beats trimming the prompt on almost every real workload.
Three buckets, three rates
Modern LLM APIs do not bill tokens at a single price. They expose at least three tiers, and the gap between them is large.
- Cached input tokens, here $0.30 per million. These are tokens the provider already processed on a previous call and kept warm, so re-processing is cheap.
- Fresh input tokens, here $3 per million. These are new prompt tokens the model must run through a full prefill pass.
- Output tokens, here $15 per million. These are tokens the model generates, one at a time, during decode.
The rates span a 50 to 1 range from cheapest to dearest. That spread is the entire point of the question. A correct cost estimate requires keeping the buckets separate and billing each at its own rate. Blending them into one average rate, or worse, applying the input rate to everything, produces a wrong answer and hides the lever that actually controls spend.
It also pays to notice what falls into each bucket on a real request. The cached bucket is your stable scaffolding: a long system prompt, tool and function schemas, retrieval instructions, and few-shot examples that never change between calls. The fresh input bucket is whatever is unique to this turn: the user's question and the retrieved chunks a RAG pipeline just fetched. The output bucket is the model's reply. Classifying every token into the right bucket before you multiply is half the skill, because a token in the wrong bucket can be off by 10x or 50x in price.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Anthropic's Claude prompt caching cuts cached input to roughly a tenth of the standard input rate, the discount modeled in this calculation.
- OpenAI's automatic prompt caching applies a similar cached-input discount on GPT-5.5, making long static system prompts cheap to reuse.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy is the output rate so much higher than the input rate?
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating all tokens as one price, or assuming the 10,000-token prompt must dominate because it is biggest. The output line wins on price per token, not on count.
60 second bullets to scan on the way to the call.
The three token buckets and the distinct rate each one carries
How to compute one line item as count times rate over one million
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.