Explain how reasoning models can charge for tokens the caller never sees in the response. Cover what those hidden tokens are, why they cost real money, and how providers typically surface them in usage accounting.
Free · 2 AI evals / day
**Reasoning models emit a hidden chain-of-thought.** Systems like the OpenAI o-series, Anthropic Claude Opus 4.7 with extended thinking, and DeepSeek V4 are trained to generate internal reasoning tokens before they produce the visible answer. Those reasoning tokens are real generated tokens, sampled by the same decode loop as the final reply.
Each reasoning token costs the same GPU time as a visible token. The forward pass, the KV-cache read from HBM, the matmul over the parameters: all of it runs identically. The model just chooses (because of how it was trained) to mark some output as reasoning and some as the final answer, and the serving runtime hides the reasoning from the response payload by default.
Providers bill for the GPU work, so reasoning tokens bill as output (completion) tokens, at the higher of the two pricing tiers. The standard pattern is to surface them in the usage object: OpenAI uses `completion_tokens_details.reasoning_tokens`, DeepSeek uses `thinking_tokens`, Anthropic uses an extended-thinking counter. Total billed output equals visible completion tokens plus reasoning tokens.
Two practical implications. First, set output budget caps (`max_completion_tokens` or equivalent). Without one, a single request can run for tens of thousands of reasoning tokens and cost orders of magnitude more than you expected. Second, when comparing reasoning models to non-reasoning ones on cost, always compare total billed completion tokens, not the visible answer length, or you will reach the wrong conclusion.
**The hidden tokens are a chain-of-thought, and they bill as output.** Reasoning-tuned models (OpenAI o-series, Claude with extended thinking, DeepSeek V4) generate a long internal reasoning trace before producing the visible answer. Each reasoning token is a normal decode step: full forward pass, full KV-cache read, full HBM bandwidth load. The serving runtime hides the trace from the response payload, but the GPU did the work.
Providers bill for the GPU work. Reasoning tokens count as completion (output) tokens, at the same price as visible output. Total billed output equals visible_tokens plus reasoning_tokens, and the reasoning side typically dominates: a 50-token visible reply can hide 5,000 reasoning tokens behind it, so the bill is 100x what the visible length would suggest.
Usage objects surface this. OpenAI exposes `completion_tokens_details.reasoning_tokens`. DeepSeek exposes a `thinking_tokens` counter. Anthropic exposes the same idea under extended-thinking accounting. Production dashboards that only watch `completion_tokens` undercount badly once a workload moves to a reasoning tier.
Two practical defenses. **Set output-side budget caps** like `max_completion_tokens` or an explicit reasoning-token cap; without one, a single request has no upper bound on cost. **Normalize cost comparisons on total billed completion tokens**, not visible answer length, when picking between a reasoning model and a non-reasoning one. The reasoning model often loses on raw cost-per-task once the hidden tokens are counted honestly. Whether the higher cost is worth it is a separate question, but it should be answered with the right numbers.
Quick reference
TL;DR
Reasoning models generate a hidden chain-of-thought before the visible answer. Those tokens cost real GPU decode time and bill as output tokens, often dwarfing the visible reply by 10-100x.
Picture a student who scribbles three pages of working notes to figure out a math problem and then writes a one-line answer on the test sheet. The teacher only sees the one line, but if they were paying the student per page of work produced, they would owe for the notes too. A reasoning model is the same. Before it writes the final answer you see, it generates a long internal scratchpad of partial reasoning, hypothesis checks, and self-corrections. The provider's GPU did real work for every one of those scratchpad tokens, so the bill includes them. The visible reply might be 50 tokens, but the line item on your invoice can show 5,000 output tokens because the model wrote 4,950 tokens worth of hidden thinking to get there.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Trivial completions like formatting, rewording, or surface-level rewrites The model has nothing meaningful to reason about, so reasoning tokens are pure cost without quality lift, and a non-reasoning tier is the obvious pick.
Latency-critical interactive routes with tight TTFT budgets The thinking pass adds noticeable wall-clock latency before the user sees any output, breaking interactive UX even when the dollar cost is acceptable.
Cost-sensitive workloads at high volume without per-request caps Reasoning amplification is heavy-tailed and a single uncapped request can spike to tens of thousands of output tokens, undermining unit economics.
Real products, models, and research that use this idea.
1OpenAI o-series exposes reasoning tokens in completion_tokens_details.reasoning_tokens and bills them at the standard output rate.
2Anthropic Claude Opus 4.7 with extended thinking exposes a budget_tokens parameter and surfaces reasoning tokens in the usage object alongside output_tokens.
3DeepSeek V4 returns a thinking_tokens counter in the usage block and bills them as completion tokens at the output tier.
4Coding agents that switched to reasoning-tier backends in 2026 report 10-30x monthly output cost growth that resolves once they add a difficulty router and per-request reasoning caps.
5Cost monitoring vendors like Helicone and LangSmith added reasoning-token dashboards in 2026 specifically because the old completion-token charts missed the new cost driver.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you instrument cost monitoring to catch reasoning-token regressions early?
Sign in or go Pro to see the approach for this follow-up.
QWhat knobs do providers expose to cap reasoning, and what is the tradeoff?
Sign in or go Pro to see the approach for this follow-up.
QHow does prefix caching interact with reasoning outputs?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Comparing reasoning models to non-reasoning models on the length of the visible answer. The reasoning model looks cheap by that metric and ruinously expensive on the actual bill, because thinking tokens dominate output count.