Your product serves four request classes: (a) FAQ deflection, (b) text to SQL over a 40 table warehouse, (c) 80 page contract risk review, and (d) ad hoc analytics questions with numeric answers. Design a reasoning effort policy: assign an effort tier to each class, and name the two classes you would monitor most closely after launch.
FAQ no reasoning, text to SQL low or medium with execution verifier, contract review high async, analytics medium with self-consistency; monitor the two verifiable classes hardest.
Imagine your product is a busy kitchen with one head chef who can think for hours about hard dishes and a line cook who can plate simple orders in seconds. You would not send a glass of water to the head chef; that wastes their time and slows the customer. You would not send a 12-course tasting menu to the line cook; that one needs deep planning. Some orders are in between: a steak temperature needs care but not an hour of thinking. The smart kitchen routes each ticket to the right station based on how hard the dish is and how long the customer is willing to wait, and tastes its own work most often on the dishes where it can actually check the result.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Effort-tier policy questions are common in 2026 staff and senior screens because most teams are simultaneously trying to use reasoning models for the workloads that benefit from them and avoid using them on the workloads that do not. The default fail mode is to set one global effort tier (usually medium, sometimes high) and let cost run away on easy traffic while quality still lags on hard traffic. The correct policy is per-class, justified by the shape of each class's cost-quality curve.
The four classes in this question are deliberately chosen to cover the four quadrants of (task hardness × output verifiability). FAQ is easy and unverifiable in a free-form sense. SQL is medium and executable. Contract review is hard and judgment-based. Analytics is medium and numerically checkable. A good policy treats each one differently, and a great policy uses the verifiable classes as the empirical signal that tunes everything else.
Per-class effort assignment with mechanism
FAQ deflection: no reasoning. The cost-quality curve flatlines almost immediately. A fast non-reasoning model (Claude Haiku 4.5, GPT-5-mini, Gemini Flash 3) hits the quality ceiling for templated answers; additional thinking tokens add latency without measurably better outputs. The right route is a small model with retrieval over the FAQ knowledge base.
Text to SQL: low or medium effort with execution-based verifier. The cost-quality curve climbs but the climb is dominated by structural errors (missing joins, wrong columns) that an execution verifier catches cheaply. The policy: generate at low effort, execute (or use a dry-run plan check) against the warehouse, retry at medium effort on verifier failure, escalate to high only on second failure. This three-tier ladder bounds worst-case cost at three calls while keeping the common case at one cheap call.
Contract review: high effort, async path. The cost-quality curve climbs steeply with effort because every additional thinking pass picks up risks the previous pass missed. The user is uploading a document and accepting a multi-minute wait, so synchronous latency is not a constraint. The policy: highest available thinking budget, surface a progress UI, design for chunked processing if the document exceeds context.
Ad-hoc analytics: medium effort with self-consistency. The output is an exact number, which means samples can be majority-voted. Self-consistency at medium effort (three to five samples, majority vote on the number) is empirically competitive with high effort at one sample, at roughly half the cost. The policy: medium effort, n=3 samples, majority vote, escalate to high effort if no majority emerges.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Anthropic's Claude Opus 4.7 extended thinking with explicit token budgets is the pattern for the contract-review tier; the API exposes the budget knob directly.
- OpenAI o-series reasoning effort low or medium or high is used in production routers like Cursor and Continue to pick effort per query class.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you decide when to add a fifth tier or collapse two existing tiers?
Add a tier when an existing class's accuracy curve has a clear elbow that current tiers do not capture; collapse tiers when two adjacent ones show statistically indistinguishable verifier pass rates and cost differences.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Setting one global effort tier across all classes; the cost-quality curve is different per class and the right policy is per-route.
60 second bullets to scan on the way to the call.
Why FAQ traffic should not touch a reasoning model
How execution-based verification lets you lower SQL effort safely
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.