You front a cheap fast model and an expensive reasoning model with a router. Design it: which signals decide the initial route, what the escalation path looks like when the fast model whiffs, and how you would prove the router itself is not the new bottleneck.
Classifier and prompt features pick the tier; draft, verify, escalate on low confidence; prove the router by overhead, escalation rate, and bidirectional misroutes against cost per resolved query.
Think of a triage nurse at a hospital. They glance at each patient and send the easy cases to a general practitioner who works fast and cheap, and the complicated cases straight to a specialist. Sometimes the nurse is wrong: a patient looks simple but gets worse, so the GP escalates them to the specialist. Sometimes the nurse is wrong the other way: a patient is sent to the specialist but really just needed a band-aid, and the specialist's time is wasted. A good hospital tracks both kinds of mistakes and checks that the nurse herself is not the slowest part of the visit. That is exactly what a fast model and reasoning model router has to do.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Routing between a fast model and a reasoning model is one of the most-discussed system design probes for senior LLM engineers in 2026, because every team that deploys reasoning models at scale eventually builds one. The naive version is a classifier in front of two endpoints. The version that actually ships is a draft then verify then escalate cascade with explicit metrics that catch misroutes in both directions.
This explanation walks the three layers of routing signal, the verifier patterns that make escalation safe, the metrics that prove the router is doing its job, and the conditions under which the architecture earns its engineering cost.
Three layers of routing signal
Pre-request signals. Available before any inference runs. Prompt length, presence of structured content (code blocks, tables, math notation), explicit task markers in the request metadata, user tier (paid users get reasoning headroom, free users default to fast), endpoint URL (a contract-review endpoint defaults differently than a chat endpoint). These signals are cheap and feed a lightweight classifier or rule set. They are sufficient to make a default decision for the easy majority of traffic.
In-request signals. The fast model's draft plus its self-reported confidence or token-level uncertainty. Available immediately after the fast call returns. Fast models are imperfectly calibrated in absolute terms (a fast model saying 'high confidence' does not always mean correct), but they are useful in relative terms (the same model is more confident on easy queries than hard ones within a task class). Some teams also use the entropy of the model's logits as a confidence proxy.
Post-draft signals. Verifier output. The verifier can be an execution check (run the SQL, compile the code, evaluate the math), a judge model (a small LLM that scores the draft against a rubric), or a deterministic rule check (regex, schema validation, length bounds). The verifier is the load-bearing piece of the cascade because it converts the soft 'maybe the draft is wrong' signal into a hard 'escalate or do not' decision.
Layer ordering matters. Cheap signals first, expensive signals only as needed. Pre-request signals make the initial route; in-request signals catch obvious problems for free; post-draft signals catch the rest at the cost of one verifier call.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Anthropic's Claude routing in agent frameworks like Claude Code uses Haiku 4.5 as the fast tier and escalates to Sonnet or Opus 4.7 thinking on harder tasks.
- OpenAI's GPT-5 routing between mini and full o-series models in production tools uses verifier-style scoring for code and math.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you choose between an execution verifier, a judge model, and a self-confidence signal for a new task class?
Pick execution if the output is checkable mechanically (code, SQL, math); pick judge if the output is free-form but has rubrics; pick self-confidence only as a complement, since fast models are often miscalibrated in absolute terms but useful for relative ordering.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating the router as a one-shot classifier instead of a draft then verify pipeline; the verifier is what makes the architecture safe at low cost.
60 second bullets to scan on the way to the call.
Which signals feed the initial route decision
Why a verifier is required to make escalation safe
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.