Estimate the monthly bill before and after adding a model routing tier
An endpoint serves 2,000,000 calls per month. Every call sends 4,000 input tokens and receives 500 output tokens. The current model costs $2.50 per 1M input tokens and $10.00 per 1M output tokens. A proposed routing tier would send 80 percent of calls to a smaller model priced at exactly one tenth of those rates ($0.25 and $1.00 per 1M), with the remaining 20 percent staying on the big model. Compute the monthly bill (a) today and (b) after routing.
$30,000 before, $8,400 after. The big-model 20 percent of traffic still drives $6,000 of the post-routing bill, so escalation accuracy dominates the economics.
Imagine a coffee shop where every customer buys a fifteen-dollar espresso. You sell two million espressos a month and your bill is thirty thousand dollars. Then you notice that most customers would be perfectly happy with a one dollar fifty regular coffee instead. You build a quick survey at the door: cheap drink for the eighty percent who do not need the fancy one, espresso for the twenty percent who do. Your monthly bill drops to eighty four hundred. The interesting part is that the twenty percent who still buy espresso account for six thousand of that new bill. So the trick is not making the cheap drink cheaper. It is making sure the door survey gets the boundary right.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Model routing is one of the highest-leverage cost optimizations available to LLM-powered products in 2026, and this problem isolates the math that makes it work. The mechanical part (multiply, sum, subtract) is what the interview asks for. The interesting part is the structural observation about where the new bill concentrates, because that observation tells the team where to invest follow-up engineering effort.
This deep dive walks the per-call decomposition, the weighted sum after routing, the structural observation about the expensive bucket, and the strategic implications for router design, escalation tuning, and contract negotiation.
Per-call cost decomposition
Per-call cost on any LLM is input cost plus output cost, each priced per million tokens. For the big model in this problem:
A subtle observation: input is two thirds of the per-call cost despite being eight times the token count of output. That is the standard ratio in 2026 RAG-style workloads where prompts are long (retrieved context plus instructions) and responses are short.
Monthly bill before routing: 2,000,000 x $0.015 = $30,000.
This baseline matters because it sets the denominator for the routing savings calculation. Without computing the baseline carefully (separating input and output rates), the savings number is wrong by a lot.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenRouter, LiteLLM, and Portkey expose model routing tiers with per-call cost reporting and escalation policy
- Anthropic Claude Sonnet 4.6 paired with Haiku 4 follows roughly the 1:10 cost ratio assumed here for general traffic
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat does the bill look like if the router moves the 80/20 split to 90/10?
Recompute: 1,800,000 x $0.0015 + 200,000 x $0.015 = $2,700 + $3,000 = $5,700. The expensive bucket drops by half and the bill drops by another third. This is the lever that pays.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Computing the savings as a flat ten-times reduction and getting closer to $3,000. The 20 percent of expensive traffic survives at full price and dominates the new bill.
60 second bullets to scan on the way to the call.
How to decompose per-call cost into input and output components
How to compute the weighted post-routing bill from per-bucket rates
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.