Define test time compute and the accuracy tradeoff it enables.
Test-time compute is extra inference work — longer CoT, self-correction, best-of-N, or search — traded against latency and cost for higher accuracy on hard queries.
Training-time compute builds the brain during study. Test-time compute is extra thinking during the exam after you read the question — working longer, trying multiple approaches, or checking your work. Reasoning models deliberately spend more of this per query to get harder problems right, but each extra second and token shows up on the bill.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Reasoning models are often introduced as 'models that think longer.' Test-time compute is the precise term for that idea — and for the serving economics interviewers expect you to articulate.
This flashcard anchors the definition before harder questions on when inference scaling beats training scaling.
Definition and lifecycle placement
Test-time compute is FLOPs and tokens spent during inference on a live query, after weights are frozen.
Pretraining and post-training RL consume training-time compute — they shape the checkpoint once. Test-time compute spends resources per request to extract better answers from that checkpoint.
The flashcard back lists the canonical forms: longer generations, self-correction, best-of-N, and search. All share the property that the user (or batch job) waits longer and the provider bills more tokens or GPU-seconds.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenAI o1/o3 meter hidden thinking tokens separately from user-visible completions — classic test-time compute productization.
- DeepSeek-R1 extends decode length with long internal reasoning traces before final answers.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does adaptive thinking budget routing work in production?
Classifier or heuristic routes easy prompts to fast single-sample path; hard verifiable prompts get extended thinking or best-of-N — contain tail cost.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Defining test-time compute as more pretraining or a bigger GPU — it is per-query inference spend after the prompt arrives.
60 second bullets to scan on the way to the call.
Define test-time compute in one sentence
Contrast with training-time compute
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.