Estimate the per call cost of a typical RAG chatbot using GPT-4o-mini.
Input 1700 tokens at 0.15 USD per million plus output 250 tokens at 0.60 USD per million is roughly 0.0004 USD per call, about 1,200 USD per month at 100k calls per day.
Imagine paying for a taxi by two meters at once, one ticking on every kilometre you ride in (input) and a second ticking on every kilometre the driver speaks an answer (output). The output meter ticks four times faster per kilometre, but in this trip the input is much longer (about seven times the output) because the retrieved documents take up most of the ride. So even though the output rate is higher, the input still ends up costing more per trip. Add the two meters and one taxi ride costs a tiny fraction of a cent. Run a hundred thousand rides a day and the bill quietly climbs to about forty dollars.
Detailed answer & concept explanation~6 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
3 min: per call math + input vs output asymmetry + retrieved-context dominance + scaling to daily and monthly + prompt-caching and rerank levers.
Real products, models, and research that use this idea.
- OpenAI's pricing page lists GPT-4o-mini at 0.15 USD input and 0.60 USD output per million tokens in 2026, with prompt caching cutting input prices roughly in half on cached blocks.
- Anthropic Claude Haiku 4 (the closest equivalent in capability) sits at similar price points with batch-API discounts of 50 percent.
- Vercel AI SDK ships usage tracking that surfaces input vs output token cost per request for cost monitoring.
- Modal Labs and Together.ai publish self-host cost calculators that compare commercial API spend against fine-tuned Llama 4 8B serving at scale.
- Helicone and Langfuse provide observability dashboards that decompose RAG costs by retrieved-token volume, which is the lever most teams hit first.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat if context grows to 5,000 tokens (heavy reranker dropped)?
QWhen would self-hosting Llama 4 8B beat GPT-4o-mini economically?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Quoting one combined price for input and output. They are priced separately, input is cheaper per token, but RAG context inflates input volume far above output.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.