Estimate the daily savings from dropping default high reasoning effort
Your service defaults every request to high reasoning effort, averaging 6,000 reasoning tokens per request. Profiling shows that for 80% of your 1,000,000 daily requests, low effort (averaging 800 reasoning tokens) keeps accuracy flat. Reasoning tokens bill at $10 per 1M output tokens. Estimate the daily savings from switching that 80% slice to low effort.
Switching 800,000 daily requests from 6,000 to 800 reasoning tokens saves 4.16B tokens at $10 per million, or $41,600 per day, with no accuracy loss on the affected slice.
Imagine a delivery company that sends a full eighteen-wheeler for every single order, including a single envelope. The trucks cost the same regardless of cargo. Then a profiler shows that 80 percent of orders fit in a bicycle bag. Switching that slice to bikes saves a fortune in fuel without changing what gets delivered, because the bicycle was always enough for those orders. Same idea for a reasoning service that defaults every request to maximum thinking. Most requests do not need it. Right-sizing the thinking budget for the easy slice claws back tens of thousands of dollars a day.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Default-high reasoning effort is the most common cost trap on a 2026 reasoning-model deployment. The integration ships with the maximum effort setting because that is what passed integration tests; the default lives quietly until the first end of month invoice arrives 5 to 10 times higher than the load-test projection. The diagnosis is almost always the same: the workload has a large easy slice that was paying the deep-tier price for no measurable accuracy gain.
This deep dive walks through the savings arithmetic in the worked example, the tiering architecture that captures the savings in production, the escalation pattern that preserves the deep-tier ceiling on the requests that benefit, and the verification practices that prevent the cost cut from silently regressing accuracy.
The arithmetic, step by step
Four steps, each isolated.
Step 1: affected requests
Eighty percent of 1,000,000 daily requests is 800,000. The other 200,000 stay on the deep tier; only the easy slice is touched.
Step 2: tokens saved per request
The deep tier averages 6,000 reasoning tokens per request. The low tier averages 800. Savings per request is 6000 - 800 = 5200 tokens.
Step 3: total tokens saved per day
Multiply: 800{,}000 * 5{,}200 = 4{,}160{,}000{,}000 tokens, or 4.16 billion reasoning tokens per day.
Step 4: convert to dollars
Pricing is \$10 per 1 million output tokens. 4.16 billion tokens is 4,160 million tokens. Cost: 4{,}160 * \$10 = \$41{,}600 per day.
A compact one-liner
with the worked values: 1{,}000{,}000 * 0.8 * 5200 * 10 / 1{,}000{,}000 = 41{,}600.
Sanity-checking the answer
The affected requests are 800k. Tokens saved are about 5k each. Rate is roughly \$1 per 100k tokens. So 800{,}000 * 5{,}000 / 100{,}000 = \$40{,}000, close enough to confirm the precise answer of \$41{,}600. Order of magnitude checks like this catch the most common unit error: dividing by 1,000 instead of 1,000,000 and arriving at \$41.6M per day.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenAI o-series exposes a reasoning_effort parameter (low, medium, high) so per-class tiering is a single-field change per request
- Anthropic Claude Opus 4.7 extended thinking exposes a budget_tokens parameter that supports the same per-class tiering pattern
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you build the per-class router so the accuracy on the cheap tier is provably stable?
Define request classes from prompt features (length, task type, retrieval depth, presence of math). Sample 1-2 percent of traffic per class for a daily eval run at both tiers; alert if the cheap-tier accuracy drops by more than a threshold. The router itself runs on cheap rules or a small classifier with sub-millisecond latency.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Multiplying everything (including unaffected 20 percent) or applying the wrong unit (million versus thousand) and arriving at a number that overstates or understates by 10x or more.
60 second bullets to scan on the way to the call.
How to compute reasoning-token savings from a per-class profile
Why a default-high effort policy is a silent cost multiplier
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.