Chain-of-Thought is widely cited as 'always improves accuracy'. For which of these tasks would adding CoT prompting most reliably help?

Chain-of-thought prompting asks the model to write out its reasoning before giving the final answer. The empirical finding is clear: CoT helps a lot on tasks that involve multiple steps and barely helps (or hurts) on tasks that do not. A multi-step word problem with arithmetic, intermediate values, and conditional logic is the canonical case where CoT shines. Each intermediate step is a checkpoint that prevents the model from collapsing the whole computation into one guess. A single-fact lookup like the capital of France is the opposite case. The answer is one token away from the prompt. Asking the model to think step by step adds tokens and latency, and occasionally introduces an invented reasoning chain that drifts away from the right answer. The rule of thumb is simple. Use CoT when the task has steps to take. Skip it when the task does not.

Chain-of-thought is one of those patterns that gets oversold. The original 2022 papers showed large accuracy gains on math word problems, commonsense reasoning, and symbolic reasoning. The same papers also showed near-zero gains on tasks where the answer is a direct retrieval from training data. The mechanism is mechanical. Producing intermediate tokens lets the model spend compute on subproblems. For a multi-step word problem, each intermediate value (the running total, the rate, the partial answer) becomes a checkpoint that anchors the next step. For a single fact lookup, there are no subproblems, so the extra tokens are pure overhead. The practical heuristic is task structure, not task difficulty. A simple task with two real steps benefits from CoT. A complex-looking task that is actually one retrieval does not. In production, this means CoT belongs in the system prompts of multi-step pipelines such as planning, calculation, and multi-hop QA. It does not belong in the prompts of routine extraction, translation, or fact-lookup paths. The corollary is that universal CoT is an antipattern. Adding it to every prompt burns tokens and latency on traffic that does not benefit, and can occasionally invent a wrong chain of reasoning that misleads the final answer.