Estimate the per call cost of a typical RAG chatbot using GPT-4o-mini.
Input 1700 tokens at 0.15 USD per million plus output 250 tokens at 0.60 USD per million is roughly 0.0004 USD per call. That is about 1,200 USD per month at 100k calls per day.
Imagine paying for a taxi with two meters at once. One ticks on every kilometre you ride in (input). A second ticks on every kilometre the driver speaks an answer (output). The output meter ticks four times faster per kilometre. But in this trip the input is much longer (about seven times the output), because the retrieved documents take up most of the ride. So even though the output rate is higher, the input still ends up costing more per trip. Add the two meters and one taxi ride costs a tiny fraction of a cent. Run a hundred thousand rides a day and the bill climbs to about forty dollars.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
LLM cost arithmetic is concrete and verifiable. The numbers here are simple enough to do in your head with practice. The structural lessons (input dominates RAG cost, retrieved context inflates the bill) generalise across every commercial API.
An exact dollar figure matters less than the method. Decompose the call into input and output, apply the asymmetric pricing, and see that retrieved context dominates input volume. Then project daily and monthly cost at realistic traffic and name at least one optimisation lever.
The per call arithmetic
All major commercial APIs price input and output separately. GPT-4o-mini is 0.15 USD per million input tokens and 0.60 USD per million output tokens, a 4x asymmetry.
In this scenario, the input total is 200 (user prompt) + 1500 (retrieved context) = 1700 tokens. Per-call input cost is 1700 * 0.15 / 1,000,000, which gives 0.000255 USD.
Output is 250 tokens. Per-call output cost is 250 * 0.60 / 1,000,000, which gives 0.00015 USD.
Total per call: 0.000255 + 0.00015 = roughly 0.0004 USD. That's 4 hundredths of a cent.
Even though output is priced 4x higher per token, input is 6.8x larger by volume, so input cost dominates by 1.7x. The ratio of retrieved-context volume to response volume sets the cost ratio of any RAG call.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenAI's pricing page lists GPT-4o-mini at 0.15 USD input and 0.60 USD output per million tokens. Prompt caching cuts input prices roughly in half on cached blocks.
- Anthropic Claude Haiku 4.5 (the closest equivalent in capability) costs several times more per token, with batch-API discounts of 50 percent.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat if context grows to 5,000 tokens (heavy reranker dropped)?
Input becomes 5,200 tokens at 0.15 per million = 0.00078 USD. Output stays 0.00015 USD. Per-call doubles to roughly 0.001 USD. At 100k calls per day, that's 100 USD per day, 2.5x the original.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Quoting one combined price for input and output. They are priced separately. Input is cheaper per token, but RAG context inflates input volume far above output.
60 second bullets to scan on the way to the call.
Input versus output pricing asymmetry for GPT-4o-mini
How to compute per call cost from token counts
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.