Flashcard: what is chain-of-thought (CoT) prompting?

Flashcard·Easy·4.0 · 0·~30s·Asked atAccenture Turbopuffer Writer·Relevant atAnthropic

Attempt it

Chain-of-thought prompting asks the model to write out its intermediate reasoning steps before producing the final answer, typically by adding a phrase like 'Let's think step by step' (zero-shot CoT) or by showing few-shot examples that demonstrate the reasoning trace. It improves multi-step problem accuracy because each intermediate step becomes a sub-problem the model can solve sequentially, with prior steps available as context for later steps. Without CoT, the model has to predict the final answer in one shot, which fails on math, logic, or multi-hop reasoning where humans also rely on scratch work. CoT is most useful on tasks that decompose naturally into sub-steps (arithmetic word problems, logical deduction, multi-hop QA); it adds little value on simple recall or pattern-matching tasks. The cost is longer output (more tokens, more latency, more dollars). Modern reasoning models like OpenAI o3 and Claude Opus 4.7 extended-thinking mode bake CoT into the model's hidden reasoning loop, but the user-facing prompt-level CoT pattern is still widely used.

Your answer

**Chain-of-thought prompting** (CoT) is the technique of asking the model to write out its reasoning before giving the final answer. The simplest form is to add the phrase 'Let's think step by step' to your prompt. The model then produces a sequence of intermediate reasoning steps and only at the end produces the answer. Why this works. The model generates output one token at a time, and each new token uses all the prior tokens (input plus already-generated output) as context. When the model writes out reasoning first, those reasoning steps become part of the context for the final answer. The model effectively solves a multi-step problem one piece at a time, with each piece building on the prior pieces. Where CoT helps most: math word problems, logical deduction, multi-hop question answering, and any task that decomposes into sub-steps. Where CoT helps least: simple recall (what is the capital of France), pattern-matching tasks, and short factual lookups. The cost is more output tokens, which means more latency and more dollars. So CoT is a tool to reach for when the task needs it, not a default for every prompt.

**Chain-of-thought prompting** (CoT) is the pattern of having the model write out intermediate reasoning steps before producing the final answer. Two common forms exist. Zero-shot CoT triggers reasoning by appending a phrase like 'Let's think step by step' to the prompt. Few-shot CoT shows the model 3 to 8 worked examples where the reasoning trace is fully written out, and the model learns to follow the pattern on the new input. The mechanism is straightforward once you remember how autoregressive generation works. Each output token is conditioned on every prior token, including earlier output tokens. When the model writes out reasoning, those reasoning tokens become context for the final-answer tokens. A complex multi-step problem becomes a sequence of single-step predictions, each grounded in the prior steps. The model does not have to compute the entire answer in one forward pass. The accuracy gain is dramatic on multi-step reasoning tasks. The original 2022 paper showed PaLM-540B going from 17 percent to 56 percent on GSM8K math word problems with CoT. Modern frontier models (Claude Opus 4.7, GPT-5.5, Gemini 3.1) get most of this gain by default because they were trained on reasoning data, but explicit CoT in the prompt still helps on harder problems. Where CoT does not help. Simple recall (capital of France), single-step lookups, pattern matching, and any task where the answer is one short token. There CoT just adds latency and tokens without improving accuracy. The senior framing is that CoT is a multi-step reasoning tool, not a universal quality booster.

TL;DR

Chain-of-thought prompting asks the model to write out its reasoning steps before the final answer, decomposing multi-step problems into sequential sub-problems each step can build on.

Memory aid

Easy to grasp

Imagine asking a student to solve 23 times 47 in their head and just shout the answer. Most people would get it wrong. Now ask the same student to show their working: 23 times 40 plus 23 times 7, then add them. Far more likely to be correct. The model works the same way. When you force it to write out the steps before the final answer, each step becomes input for the next step, and the model only has to do one small piece of reasoning at a time. The magic phrase that triggers this is something like Let us think step by step or Show your work first. The trick was discovered around 2022 and has become a default pattern for any multi-step problem.

Key concepts

Chain-of-thought Reasoning Few-shot prompting

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Chain-of-thought prompting is the technique that turned LLMs from pattern-matchers into multi-step reasoners. The discovery in 2022 was small in form (add a phrase, show some examples) but large in consequence: tasks that were unsolvable for the largest models suddenly became solvable. The technique now underpins most agent loops, RAG pipelines, and reasoning workflows in production.

This deep dive defines CoT precisely, explains why it works mechanistically, walks through the variants (zero-shot, few-shot, self-consistency, tree of thoughts), and connects prompt-level CoT to the built-in reasoning modes of frontier 2026 models.

What CoT is and why it works

Chain-of-thought prompting asks the model to write out intermediate reasoning steps before producing the final answer. The simplest version (zero-shot CoT) is to append a phrase like 'Let's think step by step' to the prompt. The model then produces a sequence of reasoning tokens (the chain of thought) and finally produces the answer.

Why it works traces back to autoregressive generation. An LLM produces output one token at a time. Each token is predicted from the joint distribution conditioned on every prior token, including both the input prompt and the output tokens already generated. When the model writes reasoning before the answer, the reasoning tokens become part of the context for the final answer tokens. The model has effectively given itself scratch work.

For a hard multi-step problem (a math word problem, a logical deduction, a multi-hop QA query), the answer in one forward pass requires holding the entire reasoning state in the model's internal activations. The model has to compute the answer end to end without writing anything down. For sufficiently complex problems, this exceeds what the model can do in one pass and the answer is wrong.

With CoT, each intermediate result is materialized as output tokens. The model only has to do one small reasoning step at a time, with all prior steps available as input. The hard end to end computation becomes a sequence of much smaller sub-computations, which the model can solve sequentially.

The variants and when to use each

Where CoT does not help

Prompt-level CoT vs built-in reasoning modes

Operational pitfalls

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

The original 2022 CoT paper from Wei et al. at Google Brain showed PaLM-540B going from 17 to 56 percent on GSM8K math word problems with explicit CoT prompting.
Kojima et al.'s 2022 zero-shot-CoT paper showed that just adding 'Let's think step by step' unlocks reasoning on InstructGPT without any few-shot examples.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does self-consistency CoT differ from plain CoT, and what does it cost?

Self-consistency samples N reasoning chains at temperature > 0 from the same CoT prompt and majority-votes the final answer. Costs N times more output tokens than plain CoT but reduces variance on hard math and logic problems.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Using CoT on every prompt as a default, including simple recall tasks where it just adds latency and cost without improving accuracy.

60 second bullets to scan on the way to the call.

What CoT is (write reasoning steps before the final answer)
Two main forms (zero-shot trigger phrase vs few-shot worked examples)

Primary sources. Browse if you want the original framing.