Why can a 50-token reply from a reasoning model still bill you for thousands of tokens?

Short answer·Medium·4.0 · 0·~3 min·Asked atOpenAI

Attempt it

Explain how reasoning models can charge for tokens the caller never sees in the response. Cover what those hidden tokens are, why they cost real money, and how providers typically surface them in usage accounting.

Free · 2 AI evals / day

**Reasoning models emit a hidden chain-of-thought.** Systems like the OpenAI o-series, Anthropic Claude Opus 4.7 with extended thinking, and DeepSeek V4 are trained to generate internal reasoning tokens before they produce the visible answer. Those reasoning tokens are real generated tokens, sampled by the same decode loop as the final reply. Each reasoning token costs the same GPU time as a visible token. The forward pass, the KV-cache read from HBM, the matmul over the parameters: all of it runs identically. The model just chooses (because of how it was trained) to mark some output as reasoning and some as the final answer, and the serving runtime hides the reasoning from the response payload by default. Providers bill for the GPU work, so reasoning tokens bill as output (completion) tokens, at the higher of the two pricing tiers. The standard pattern is to surface them in the usage object: OpenAI uses `completion_tokens_details.reasoning_tokens`, DeepSeek uses `thinking_tokens`, Anthropic uses an extended-thinking counter. Total billed output equals visible completion tokens plus reasoning tokens. Two practical implications. First, set output budget caps (`max_completion_tokens` or equivalent). Without one, a single request can run for tens of thousands of reasoning tokens and cost orders of magnitude more than you expected. Second, when comparing reasoning models to non-reasoning ones on cost, always compare total billed completion tokens, not the visible answer length, or you will reach the wrong conclusion.

**The hidden tokens are a chain-of-thought, and they bill as output.** Reasoning-tuned models (OpenAI o-series, Claude with extended thinking, DeepSeek V4) generate a long internal reasoning trace before producing the visible answer. Each reasoning token is a normal decode step: full forward pass, full KV-cache read, full HBM bandwidth load. The serving runtime hides the trace from the response payload, but the GPU did the work. Providers bill for the GPU work. Reasoning tokens count as completion (output) tokens, at the same price as visible output. Total billed output equals visible_tokens plus reasoning_tokens, and the reasoning side typically dominates: a 50-token visible reply can hide 5,000 reasoning tokens behind it, so the bill is 100x what the visible length would suggest. Usage objects surface this. OpenAI exposes `completion_tokens_details.reasoning_tokens`. DeepSeek exposes a `thinking_tokens` counter. Anthropic exposes the same idea under extended-thinking accounting. Production dashboards that only watch `completion_tokens` undercount badly once a workload moves to a reasoning tier. Two practical defenses. **Set output-side budget caps** like `max_completion_tokens` or an explicit reasoning-token cap; without one, a single request has no upper bound on cost. **Normalize cost comparisons on total billed completion tokens**, not visible answer length, when picking between a reasoning model and a non-reasoning one. The reasoning model often loses on raw cost-per-task once the hidden tokens are counted honestly. Whether the higher cost is worth it is a separate question, but it should be answered with the right numbers.

Quick reference

TL;DR

Reasoning models generate a hidden chain-of-thought before the visible answer. Those tokens cost real GPU decode time and bill as output tokens, often dwarfing the visible reply by 10-100x.

Memory aid

Easy to grasp

Picture a student who scribbles three pages of working notes to figure out a math problem and then writes a one-line answer on the test sheet. The teacher only sees the one line, but if they were paying the student per page of work produced, they would owe for the notes too. A reasoning model is the same. Before it writes the final answer you see, it generates a long internal scratchpad of partial reasoning, hypothesis checks, and self-corrections. The provider's GPU did real work for every one of those scratchpad tokens, so the bill includes them. The visible reply might be 50 tokens, but the line item on your invoice can show 5,000 output tokens because the model wrote 4,950 tokens worth of hidden thinking to get there.

Key concepts

Chain of thought Reasoning models Token pricing

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

4 min: hidden chain-of-thought + billed as output + usage-object fields + 10-100x amplification + observability + budget caps + difficulty routing.

Real-world examples

Real products, models, and research that use this idea.

OpenAI o-series exposes reasoning tokens in completion_tokens_details.reasoning_tokens and bills them at the standard output rate.
Anthropic Claude Opus 4.7 with extended thinking exposes a budget_tokens parameter and surfaces reasoning tokens in the usage object alongside output_tokens.
DeepSeek V4 returns a thinking_tokens counter in the usage block and bills them as completion tokens at the output tier.
Coding agents that switched to reasoning-tier backends in 2026 report 10-30x monthly output cost growth that resolves once they add a difficulty router and per-request reasoning caps.
Cost monitoring vendors like Helicone and LangSmith added reasoning-token dashboards in 2026 specifically because the old completion-token charts missed the new cost driver.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you instrument cost monitoring to catch reasoning-token regressions early?

QWhat knobs do providers expose to cap reasoning, and what is the tradeoff?

QHow does prefix caching interact with reasoning outputs?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Comparing reasoning models to non-reasoning models on the length of the visible answer. The reasoning model looks cheap by that metric and ruinously expensive on the actual bill, because thinking tokens dominate output count.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.