Which signal best decides when to escalate a query to a larger model?
Escalate on a cheap signal that correlates with query difficulty — a classifier or the small model's calibrated confidence — not on proxies like time of day or prompt length.
Imagine a triage nurse deciding who needs the specialist. A good nurse listens to the symptoms — the thing that actually predicts how sick you are. A bad nurse sends people upstairs based on what time they arrived, or how many words they used to describe their pain. Model routing is the same. The escalation signal has to track whether the question is genuinely hard. A difficulty score or the small model's own doubt works. The clock or the prompt's length doesn't — they have nothing to do with how hard the question actually is.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
A model-routing tier lives or dies by one decision made on every request: does this query escalate to the frontier model or not? Get the signal right and you capture most of the cost savings while protecting quality on the hard queries. Get it wrong and you either burn the budget or silently ship worse answers.
The interview framing here is deliberately a multiple-choice between one good signal and three tempting but flawed ones. The point isn't to memorize that 'classifier is correct' — it's to internalize the two properties any escalation signal must satisfy, and to see precisely how each distractor violates one of them.
This deep dive lays out those two properties, walks through why the difficulty-correlated signal wins, dissects each distractor's specific failure mode, and shows how the labeled-data calibration step turns a plausible signal into a trustworthy one. By the end you should be able to evaluate any proposed routing signal on the spot.
The two properties every escalation signal needs
Strip the problem down and a good escalation signal must satisfy exactly two conditions. Both are necessary; either alone is useless.
Property one: it correlates with marginal capability need. The signal has to predict whether the frontier model would give a meaningfully better answer than the cheap one. That's the entire job — separating queries that need the big model from queries that don't. A signal uncorrelated with difficulty sends queries up and down at random relative to what actually matters.
Property two: it's cheap to compute. The signal runs on every single request, including the easy majority. If computing it costs nearly as much as the frontier call it's deciding about, you've defeated the purpose — you'd be better off just calling the frontier model.
The correct answer — a lightweight classifier or the small model's confidence — hits both. The classifier is a tiny model, milliseconds and fractions of a cent, trained on labeled difficulty. Confidence is free because you're running the cheap model anyway. Every distractor fails one of these two tests, and naming which one is how you reason about routing signals you've never seen before.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Candidate signal | Correlated with difficulty? | Cheap? |
|---|---|---|
| Classifier / model confidence | Yes | Yes |
| Always escalate | Trivially yes | No — frontier price on all |
| Time of day | No | Yes |
| Prompt length vs last prompt | Weakly / noisily | Yes |
Real products, models, and research that use this idea.
- RouteLLM (LMSYS) trains a router on preference data so only queries needing a strong model escalate to the frontier tier.
- A cheap default (Gemini 3.1 Flash or GPT-5.5-mini) with classifier-gated escalation to a frontier model (GPT-5.5 or Claude Opus 4.7) is the canonical 2026 cost-tiering pattern.
What an interviewer would ask next. Try answering before peeking at the approach.
QPrompt length is a weak proxy — but is there any case where it's a reasonable first-pass signal?
Consider context-window or cost constraints, not difficulty: very long prompts may force a larger-context model regardless of hardness, which is a different reason than capability.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Choosing an escalation proxy that's easy to compute but uncorrelated with difficulty — like prompt length or time of day — so the router escalates the wrong queries.
60 second bullets to scan on the way to the call.
The two properties a good escalation signal must have
Why a classifier or calibrated confidence qualifies
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.