Why does a reasoning model like DeepSeek V4 quietly inflate the output token bill?

Short answer·Medium·4.0 · 0·~3 min·Asked atGoldman Sachs H2o Ai Mu Sigma·Relevant atAnthropic Cloudflare Groq OpenAI

Attempt it

A team switches a workload from a non-reasoning model to a DeepSeek-V4 class reasoning model. The user visible answers look the same length, but the monthly output token bill jumps roughly 10-50x. Explain the mechanism behind that, how the hidden tokens are billed, and what the right cost side mitigations are.

Free · 2 AI evals / day

**Reasoning models** are trained to emit an internal chain-of-thought before they emit the visible answer. The thinking pass is real generation: the model produces tokens one at a time, just like the final reply, through the same decode loop and the same HBM bandwidth. Those thinking tokens are billed. The provider charges per output token generated, not per token shown to the end user. So even though the UI hides the chain-of-thought, the bill still includes it. A final answer that looks 200 tokens long can hide 2,000 to 10,000 thinking tokens behind it, which is why the output-token bill jumps by 10-50x when a workload switches from a non-reasoning model to a reasoning one. The API usually exposes this directly. Look at the usage block of the response. You will find a separate counter, often called reasoning_tokens or thinking_tokens, that aggregates the hidden pass. If you only watched completion_tokens before, that counter was zero and the cost looked manageable. With a reasoning model, that counter dominates and your old monitoring misses the spike entirely. The mitigations are routing and budget control. Cap the reasoning budget per request when the task does not need deep thinking. Route only the hard tail to the reasoning tier, and use a cheaper non-reasoning model for the rest of the traffic. Watch the reasoning-token counter as a first-class metric, the same way you watch latency or error rate.

**The cost driver is the hidden chain-of-thought.** Reasoning models like DeepSeek V4 are trained to emit an internal thinking pass before the visible answer. Those tokens are generated by the same decode loop as the final reply, occupy the same HBM bandwidth, and are billed by the provider as output tokens regardless of whether the UI shows them. That is why a final answer that looks 200 tokens long can hide 2,000 to 10,000 reasoning tokens. Output-token amplification of 10-50x is typical depending on task difficulty: easy tasks barely think, hard tasks generate long internal monologues. Either way, the visible completion length tells you almost nothing about the bill. The API exposes the hidden pass explicitly. The usage block of the response typically carries a separate counter, often named reasoning_tokens or thinking_tokens, alongside the familiar completion_tokens. The two roll up into a total output-token line that drives billing. A team that only watched completion_tokens before the switch will miss the cost regression entirely until the monthly invoice arrives. Mitigations follow the framing. **Cap the reasoning budget per request** where the API exposes that knob; for easy tasks, a low budget or disabled reasoning gives you the model's quick-thinking output at non-reasoning cost. **Route by difficulty** so only the genuinely hard tail of traffic uses the reasoning tier, and the rest goes to a cheaper non-reasoning model. **Instrument the reasoning-token counter** as a first-class metric per route, so a regression is visible the day it happens, not at month-end. The teams that get this right treat reasoning tokens as a budget line, not a hidden side effect.

TL;DR

Reasoning models emit a hidden chain-of-thought before the visible answer. Those thinking tokens count as output tokens on the bill, often 10-50x the visible reply, which is why output cost explodes silently.

Memory aid

Easy to grasp

Imagine hiring a contractor who writes a thick notebook of design notes before sending you the final blueprint. You only ever see the blueprint, but you pay for every page of notes too because the contractor charges per page of work, not per page delivered. A reasoning model is doing the same thing under the hood. Before the visible reply, it generates a long internal scratchpad of thoughts, checks them, revises, and only then emits the short final answer you read. The provider's API bills you for every token the model produced, including all the scratchpad tokens the UI hides. That is why your bill can jump dozens of times even though the answer on the screen looks the same length as before.

Key concepts

Inference cost Reasoning models Frontier models

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Reasoning models changed the cost shape of LLM inference in 2025-2026. The visible answer length, which used to be a reasonable proxy for cost, became close to meaningless because the bulk of generated tokens now live in a hidden thinking pass. A team that swaps a non-reasoning model for a reasoning model without changing its observability or its routing will routinely discover a 10-50x output-token bill regression on the next monthly invoice.

This question tests whether you understand the mechanism and the operational response, not just the slogan that reasoning costs more. The mechanism is mechanical: thinking tokens are real generated tokens, billed as output, surfaced in a specific API field that most existing dashboards do not watch. The response is routing-first and budget-first: treat the reasoning tier as a per-route allocation rather than a global model choice.

The deep dive walks the mechanism end to end, explains how the bill actually surfaces the cost, and lays out the mitigation playbook in the order it usually pays off in production. By the end you should be able to estimate cost for a reasoning-model workload, design the dashboard that catches regressions before invoicing, and decide which slices of traffic justify the tier.

What the reasoning pass actually is

A reasoning model is trained, usually with reinforcement learning on a reward that scores final-answer correctness, to emit an internal chain-of-thought before the user-visible reply. The internal thoughts are real generated tokens drawn from the same vocabulary, sampled by the same decode loop, and streamed through HBM at the same per-step cost as any other generation.

The model itself does not distinguish thinking tokens from answer tokens at the architecture level; the boundary is a learned token or a wrapping tag (DeepSeek uses thinking sections, OpenAI uses reasoning summaries, Anthropic uses extended-thinking blocks). The serving runtime usually strips or hides the thinking region from the UI by default, but it can and does include them in the API response for callers who request the trace.

The upshot is simple: thinking tokens are output tokens by every metric that matters operationally. They consume KV cache, they pay decode-step latency, they ship to the response, and they show up on the bill. The UI's choice to hide them is presentation, not accounting.

How the bill surfaces them

Why amplification is heavy-tailed

Observability that catches regressions

The mitigation playbook in order of return

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

DeepSeek V4 exposes thinking_tokens in its API usage block separately from completion_tokens, so cost attribution per route is mechanical.
Anthropic's extended-thinking endpoints on Claude Opus 4.7 expose reasoning_tokens in the usage block and let callers cap the reasoning budget per request.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you design the dashboard that catches reasoning-token regressions?

Track reasoning_tokens per route, per intent class, and per customer. Alert on the ratio of reasoning to completion tokens crossing a threshold week over week. Sample the actual hidden traces for the worst-amplification routes so you can see what the model is over-thinking.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Reading only completion_tokens on the bill and missing the separate reasoning-token counter. The hidden thinking pass usually dwarfs the visible reply for this model class.

60 second bullets to scan on the way to the call.

Why reasoning models emit hidden tokens before the visible answer
How those tokens are billed and which API field surfaces them

Primary sources. Browse if you want the original framing.