Pick the routing call for a password reset question hitting your support bot
A password-reset FAQ is templated and low-stakes; send it to the fast model, save the reasoning tier for queries that actually need multi-step work.
Imagine you have two employees behind the help desk. One is a quick clerk who knows every standard answer and replies in a second. The other is a careful analyst who closes the office door, fills three whiteboards, and then comes back with the same response the clerk would have given, just slower and pricier. A password reset is a standard answer. You hand it to the clerk. You keep the analyst free for the cases that genuinely need someone to think hard, a multi-step refund dispute, a confusing bug report, an outage post-mortem. Routing in production is just deciding which question goes to which employee.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Reasoning models changed the economics of an API call. A traditional chat model bills only what it produces visibly. A reasoning model spends an extra pool of hidden thinking tokens before it commits to an answer, and those tokens bill at the output rate. A single call can quietly cost ten or twenty times its visible-token equivalent.
That changes how you route traffic. The interesting question is no longer 'which model is best?' but 'which model is best for this request?' A password-reset FAQ sits at the simplest end of the spectrum: templated, low-stakes, known answer. It is the easiest case to route correctly, and it is also the case where teams most often get routing wrong by defaulting to a single global model.
Why thinking-token spend is asymmetric
A normal chat model emits one stream: the answer. You see every token and you pay for every token. A reasoning model emits two streams: an internal trace it uses to think, and the visible answer. Both bill at the output rate. The trace can be hundreds of tokens on a simple prompt or tens of thousands on a hard one; the visible answer is usually small either way.
The consequence is that thinking tokens dominate the bill on the reasoning tier. If a request averages 500 visible tokens and 5,000 thinking tokens, more than 90% of the cost is invisible to the user. That asymmetry is fine when the thinking actually moves the answer, and very expensive when it does not.
Production routing is the lever that controls this. Send work to the reasoning tier only when the marginal thinking spend earns measurable accuracy. Send everything else to the fast model. The decision lives at the router, not inside the model.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Anthropic's Claude Opus 4.7 with extended thinking exposes a thinking-budget setting that can be turned off per request, so support bots typically disable it for FAQ intents.
- OpenAI o-series and the reasoning effort parameter let teams route by request class: minimal for greetings, low or medium for lookups, high for genuinely multi-step tasks.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you measure whether your router is sending too much traffic to the reasoning tier?
Track reasoning-tier hit rate by intent, sample escalated FAQs into eval, compute cost per intent, compare to a baseline of fast-only on the same slice.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating quality as a free knob. Maxing out effort on every request burns thinking-token spend on traffic that gains nothing and slows the whole product down.
60 second bullets to scan on the way to the call.
Explain why thinking tokens are billed and how that changes routing decisions
Identify three request classes that should never reach the reasoning tier
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.