Input: 1700 tokens × $0.15/M = $0.000255. Output: 250 × $0.60/M = $0.00015. Total: ~$0.0004 per call. At 100k calls/day that's ~$40/day or ~$1,200/month. Cost is dominated by the input side because retrieved context inflates it 7-8× over the prompt alone.
**Token pricing** on commercial LLM APIs is asymmetric: input tokens are roughly 4x cheaper than output tokens. For **GPT-4o-mini** (2026 pricing), input is 0.15 USD per million tokens and output is 0.60 USD per million.
For this question, sum the inputs and outputs separately.
Input total: 200 (prompt) + 1500 (retrieved context) = 1700 tokens. Multiply by the per token price: 1700 / 1,000,000 * 0.15 USD = 0.000255 USD per call.
Output total: 250 tokens. 250 / 1,000,000 * 0.60 USD = 0.00015 USD per call.
Grand total: 0.000255 + 0.00015 = roughly 0.0004 USD per call, often written as 4 hundredths of a cent.
Scale it up. At 100,000 calls per day, that is 100,000 * 0.0004 = 40 USD per day, or about 1,200 USD per month.
The key insight: retrieved context (1500 tokens) dominates the input volume, even though the user prompt is only 200 tokens. Cost is driven by what you stuff into the context window, not by the user's query length.
**The arithmetic** for this question, GPT-4o-mini at 2026 prices (0.15 USD input per million, 0.60 USD output per million):
Input: (200 prompt + 1500 retrieved context) = 1700 tokens. At 0.15 USD per million, that is 1700 * 0.15 / 1,000,000 = 0.000255 USD.
Output: 250 tokens. At 0.60 USD per million, that is 250 * 0.60 / 1,000,000 = 0.00015 USD.
Per-call total: roughly 0.0004 USD, about 4 hundredths of a cent.
At 100k calls per day, that is 40 USD per day or 1,200 USD per month. At 1M calls per day, 400 USD per day or 12,000 USD per month.
**The structural insight:** input dominates cost 1.7x even though output is priced 4x higher per token. The reason is volume: RAG retrieved context (1500 tokens) is 7 to 8x larger than the user prompt (200 tokens) and 6x larger than the model's response (250 tokens).
Levers to cut cost: reduce retrieved context size (rerank tighter, use a smaller top-k). Or enable prompt caching for the system prompt and any static context. Or switch to a smaller model for the retrieval-grounded subset of queries. The first lever is almost always the right one to pull first.
Quick reference
TL;DR
Input 1700 tokens at 0.15 USD per million plus output 250 tokens at 0.60 USD per million is roughly 0.0004 USD per call, about 1,200 USD per month at 100k calls per day.
Imagine paying for a taxi by two meters at once, one ticking on every kilometre you ride in (input) and a second ticking on every kilometre the driver speaks an answer (output). The output meter ticks four times faster per kilometre, but in this trip the input is much longer (about seven times the output) because the retrieved documents take up most of the ride. So even though the output rate is higher, the input still ends up costing more per trip. Add the two meters and one taxi ride costs a tiny fraction of a cent. Run a hundred thousand rides a day and the bill quietly climbs to about forty dollars.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Workloads where latency matters more than per call cost Some cost optimisations (smaller models, aggressive caching) cost latency or quality. Make the cost vs quality vs latency tradeoff explicit.
Compliance-driven self-host requirements (PHI, PCI, sovereign data) Commercial APIs may be forbidden; the cost calculus shifts to fine-tuned open-weight serving on owned infrastructure.
Extremely low volume (under 1k calls per month) Cost optimization is premature. Engineering hours cost more than tokens; ship and revisit when traffic grows.
Workloads dominated by output tokens (long form generation) The input-dominates pattern flips when output is 5,000+ tokens. Cost levers shift to output streaming, length penalties, and model routing.
Quick scan · 2–4 min
Everything important, quickly.
3 min: per call math + input vs output asymmetry + retrieved-context dominance + scaling to daily and monthly + prompt-caching and rerank levers.
Real-world examples
Real products, models, and research that use this idea.
1OpenAI's pricing page lists GPT-4o-mini at 0.15 USD input and 0.60 USD output per million tokens in 2026, with prompt caching cutting input prices roughly in half on cached blocks.
2Anthropic Claude Haiku 4 (the closest equivalent in capability) sits at similar price points with batch-API discounts of 50 percent.
3Vercel AI SDK ships usage tracking that surfaces input vs output token cost per request for cost monitoring.
4Modal Labs and Together.ai publish self-host cost calculators that compare commercial API spend against fine-tuned Llama 4 8B serving at scale.
5Helicone and Langfuse provide observability dashboards that decompose RAG costs by retrieved-token volume, which is the lever most teams hit first.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat if context grows to 5,000 tokens (heavy reranker dropped)?
Sign in or go Pro to see the approach for this follow-up.
QWhen would self-hosting Llama 4 8B beat GPT-4o-mini economically?
Sign in or go Pro to see the approach for this follow-up.
QHow does prompt caching change the math?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Quoting one combined price for input and output. They are priced separately, input is cheaper per token, but RAG context inflates input volume far above output.