Claude Sonnet (illustrative rates for this calculation): - input tokens: $3 per 1M - output tokens: $15 per 1M - cached input tokens: $0.30 per 1M A single API call has: - 10,000 cached input tokens (system prompt + tool schemas, cache HIT) - 2,000 new input tokens (this query's retrieved chunks + user question) - 600 output tokens (the assistant's reply) Compute the total cost of this single call in US cents (¢). Round to two decimal places. Then state which component (input / cached / output) dominates the cost.
Cost is a three-line sum: cached input plus new input plus output. Output tokens dominate because the output rate is 5x input and 50x cached input, even with far fewer tokens.
Imagine a print shop with three price tiers. Photocopying a page you already printed last week is almost free. Printing a fresh page costs a bit more. But writing a brand-new page by hand, word by word, is the expensive part. An API call works the same way. The huge system prompt you reuse every time is the cheap photocopy. This query's fresh context is the mid-price print. The model's reply is the hand-written page, generated one token at a time, billed at the top rate. So even a short reply can cost more than a giant cached prompt. To spend less, you do not trim the prompt first. You make the model say less.
Detailed answer & concept explanation~8 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
3 min: three rate buckets + compute each line + sum to 1.80 cents + identify output as dominant + tie to decode cost + state the cache then shorten optimization order.
Real products, models, and research that use this idea.
- Anthropic's Claude prompt caching cuts cached input to roughly a tenth of the standard input rate, the discount modeled in this calculation.
- OpenAI's automatic prompt caching applies a similar cached-input discount on GPT-5.5, making long static system prompts cheap to reuse.
- Agent frameworks like LangChain and LlamaIndex bill mostly on output across multi-step tool loops, where each step's generation is the costly line.
- RAG products stuff thousands of retrieved input tokens per call yet still see output as the dominant cost once the system prompt is cached.
- Cost dashboards in LangSmith and Helicone split spend into input, cached, and output lines precisely because the three rates differ by an order of magnitude.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy is the output rate so much higher than the input rate?
QIf you could cut cost by 30 percent, where would you start on this call?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Treating all tokens as one price, or assuming the 10,000-token prompt must dominate because it is biggest. The output line wins on price per token, not on count.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.