Flashcard: what is chain-of-thought (CoT) prompting, in one sentence?

Flashcard·Easy·4.0 · 0·~30s

Attempt it

Chain-of-thought prompting asks the model to write out its intermediate reasoning steps before producing the final answer, typically by adding a phrase like 'Let's think step by step' (zero-shot CoT) or by showing few-shot examples that demonstrate the reasoning trace. It improves multi-step problem accuracy because each intermediate step becomes a sub-problem the model can solve sequentially, with prior steps available as context for later steps. Without CoT, the model has to predict the final answer in one shot, which fails on math, logic, or multi-hop reasoning where humans also rely on scratch work. CoT is most useful on tasks that decompose naturally into sub-steps (arithmetic word problems, logical deduction, multi-hop QA); it adds little value on simple recall or pattern-matching tasks. The cost is longer output (more tokens, more latency, more dollars). Modern reasoning models like OpenAI o3 and Claude Opus 4.7 extended-thinking mode bake CoT into the model's hidden reasoning loop, but the user-facing prompt-level CoT pattern is still widely used.

Your answer

**Chain-of-thought prompting** (CoT) is the technique of asking the model to write out its reasoning before giving the final answer. The simplest form is to add the phrase 'Let's think step by step' to your prompt. The model then produces a sequence of intermediate reasoning steps and only at the end produces the answer. Why this works. The model generates output one token at a time, and each new token uses all the prior tokens (input plus already-generated output) as context. When the model writes out reasoning first, those reasoning steps become part of the context for the final answer. The model effectively solves a multi-step problem one piece at a time, with each piece building on the prior pieces. Where CoT helps most: math word problems, logical deduction, multi-hop question answering, and any task that decomposes into sub-steps. Where CoT helps least: simple recall (what is the capital of France), pattern-matching tasks, and short factual lookups. The cost is more output tokens, which means more latency and more dollars. So CoT is a tool to reach for when the task needs it, not a default for every prompt.

**Chain-of-thought prompting** (CoT) is the pattern of having the model write out intermediate reasoning steps before producing the final answer. Two common forms exist. Zero-shot CoT triggers reasoning by appending a phrase like 'Let's think step by step' to the prompt. Few-shot CoT shows the model 3 to 8 worked examples where the reasoning trace is fully written out, and the model learns to follow the pattern on the new input. The mechanism is straightforward once you remember how autoregressive generation works. Each output token is conditioned on every prior token, including earlier output tokens. When the model writes out reasoning, those reasoning tokens become context for the final-answer tokens. A complex multi-step problem becomes a sequence of single-step predictions, each grounded in the prior steps. The model does not have to compute the entire answer in one forward pass. The accuracy gain is dramatic on multi-step reasoning tasks. The original 2022 paper showed PaLM-540B going from 17 percent to 56 percent on GSM8K math word problems with CoT. Modern frontier models (Claude Opus 4.7, GPT-5.5, Gemini 3.1) get most of this gain by default because they were trained on reasoning data, but explicit CoT in the prompt still helps on harder problems. Where CoT does not help. Simple recall (capital of France), single-step lookups, pattern matching, and any task where the answer is one short token. There CoT just adds latency and tokens without improving accuracy. The senior framing is that CoT is a multi-step reasoning tool, not a universal quality booster.

Quick reference

TL;DR

Chain-of-thought prompting asks the model to write out its reasoning steps before the final answer, decomposing multi-step problems into sequential sub-problems each step can build on.

Memory aid

Easy to grasp

Imagine asking a student to solve 23 times 47 in their head and just shout the answer. Most people would get it wrong. Now ask the same student to show their working: 23 times 40 plus 23 times 7, then add them. Far more likely to be correct. The model works the same way. When you force it to write out the steps before the final answer, each step becomes input for the next step, and the model only has to do one small piece of reasoning at a time. The magic phrase that triggers this is something like Let us think step by step or Show your work first. The trick was discovered around 2022 and has become a default pattern for any multi-step problem.

Key concepts

Chain-of-thought Reasoning Few-shot prompting

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

6 min: define CoT as reasoning before answer, explain the autoregressive mechanism, contrast zero-shot vs few-shot CoT, name self-consistency and tree of thoughts variants, cover when not to use, connect to modern reasoning-mode models.

Real-world examples

Real products, models, and research that use this idea.

The original 2022 CoT paper from Wei et al. at Google Brain showed PaLM-540B going from 17 to 56 percent on GSM8K math word problems with explicit CoT prompting.
Kojima et al.'s 2022 zero-shot-CoT paper showed that just adding 'Let's think step by step' unlocks reasoning on InstructGPT without any few-shot examples.
OpenAI o3 and Claude Opus 4.7 extended-thinking mode bake CoT into the model's hidden reasoning loop, with the model generating reasoning tokens that are billed but not shown by default.
LangChain's ReAct agent and most tool-using agent frameworks rely on CoT as the reasoning substrate for the Thought-Action-Observation loop.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does self-consistency CoT differ from plain CoT, and what does it cost?

QWhy do frontier reasoning models hide the CoT tokens from the user but still bill for them?

QWhen does few-shot CoT beat zero-shot CoT, and when do they tie?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Using CoT on every prompt as a default, including simple recall tasks where it just adds latency and cost without improving accuracy.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.