Zenaique

Predict accuracy at 64k thinking tokens from a diminishing scaling curve

Predict output·Medium·4.0 · 0·~2 min·Asked atAi21CapgeminiHebbia
Attempt it
On your internal math benchmark, accuracy versus average thinking token budget looks like: 1k tokens gives 42%, 4k gives 58%, 16k gives 66%. Each 4x budget increase is buying roughly half the previous gain. If the same diminishing trend continues, roughly what accuracy do you expect at 64k tokens?
TL;DR

Each 4x budget step halves the gain (+16, +8, then +4), so 16k at 66% extrapolates to roughly 70% at 64k thinking tokens.

Memory aid
Sign in to see the mnemonic that makes this stick.
Easy to grasp

Imagine learning a new language. The first month you go from zero to holding a basic conversation, huge gain. The second month you go from basic to comfortable, smaller gain. The third month you go from comfortable to confident, smaller still. Each new chunk of effort buys less than the chunk before it. Test-time compute on a reasoning model has the same shape. The first thousand thinking tokens move accuracy a lot. The next four thousand move it less. The next sixteen thousand move it less again. So if your gains are halving with each step, you can extrapolate the next step and see roughly where things land.

Key concepts

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Test time compute scaling is the central economic story of reasoning models. The promise is that you can buy accuracy by spending more thinking tokens; the reality is that the price of each new accuracy point keeps climbing. A handful of data points is usually enough to read the shape of the curve, and that shape tells you when to stop paying for more thinking and switch strategy.

The stem hands you three data points: 1k -> 42%, 4k -> 58%, 16k -> 66%. The arithmetic on the per-step deltas yields a clean answer for 64k. The interesting part is what the shape implies about production decisions.

Reading the per-step deltas

Absolute accuracy is the wrong number to extrapolate from. The shape lives in the deltas.

From 1k to 4k tokens, accuracy moves from 42% to 58%. That is a gain of +16 points for a 4x budget increase.

From 4k to 16k tokens, accuracy moves from 58% to 66%. That is a gain of +8 points for the next 4x. The gain halved.

From 16k to 64k tokens, if the same halving pattern continues, the gain should be roughly +4 points. Add to 66% and you land near 70%.

This is a textbook saturating curve. In log-budget space (where each step is a 4x increase) the gains decay geometrically: 16, 8, 4. The continuous version of this is accuracy(budget) ~ a - b * budget^(-k) for some decay exponent k; the discrete halving per step pattern fits a curve where each multiplicative step in budget yields a multiplicatively smaller gain. The exact functional form does not matter for an extrapolation one step ahead; the pattern does.

Why this curve shape shows up everywhere on reasoning models
Translating the curve into marginal cost
What to do when you reach the bend
Extrapolation discipline
Extrapolating reasoning scaling curves carefully
Sign in to unlock the full deep dive.

Situations where this technique stops working.

Sign in to see when this approach fails.

2–4 min · Everything important, quickly.

Sign in to see the quick scan of the deep dive.

Real products, models, and research that use this idea.

  • OpenAI's 'Scaling test-time compute' research in 2024-2025 documented log-linear scaling curves with saturation on AIME and competition math.
  • Claude Opus 4.7 thinking exhibits diminishing returns past a problem-dependent budget on internal benchmarks; effort levels are calibrated to the bend.
Sign in to see more production examples.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you decide whether to spend another 4x on thinking versus switch to verifier-guided best-of-n?
A

Compare projected marginal accuracy per dollar between the two strategies; best-of-8 at moderate budget often dominates one chain at matched total compute when an oracle is available.

1 more follow-up an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Assuming the early steep gains continue. The curve flattens fast; extrapolating linearly overstates how much accuracy more compute buys you.

Sign in to see all red flags and common mistakes.

60 second bullets to scan on the way to the call.

  • Compute per-step gains from a sequence of accuracy versus budget data points

  • Recognize a log linear with decay shape from the gains

Sign in to unlock the revision sheet.

Primary sources. Browse if you want the original framing.

Similar questions

Same topic, related formats. Practice these next.

4 curated
Next question
Match each RL algorithm trait to PPO or GRPO.
Match pairs·Medium