Flashcard: why use Tree of Thoughts search on hard agent planning problems?

Flashcard·Easy·4.0 · 0·~30s

Attempt it

Tree of Thoughts (Yao et al. 2023, arXiv 2305.10601) explores multiple reasoning branches at each step instead of one linear chain. The model generates several candidate thoughts at each node, an evaluator (often the same LLM acting as judge) scores each branch, weak branches are pruned, and the search continues from the strongest. The shape is a search tree with the model as both expansion engine and value function. It solves the problem that plain chain-of-thought commits to one reasoning path token by token, with no way to backtrack if an intermediate step is wrong. Tasks where a wrong intermediate step cannot be recovered from with linear reasoning, such as puzzles (Game of 24, crosswords), planning, and constrained creative writing, are the canonical ToT wins. The cost is high: branching factor times depth model calls, often 10-100x plain CoT. Used selectively when accuracy on hard search-shaped problems matters more than cost.

Your answer

**Tree of Thoughts** (ToT) is a prompting pattern where the model explores several possible reasoning paths instead of writing one linear chain. At each step, the model generates a small set of candidate next thoughts. An evaluator (often the same LLM acting as judge) scores each candidate. The weakest candidates are pruned and the search continues from the strongest ones. The overall shape is a search tree. The root is the original problem. Each node is a partial reasoning state. Each edge is a possible next thought. The model is both the expansion engine (generating next thoughts) and the value function (scoring them). The problem ToT solves is that plain chain-of-thought commits to one path token by token. If step three goes wrong, every later step reasons from the wrong starting point. There is no way to back up. ToT lets the agent backtrack. If a branch scores poorly, it gets pruned and the search continues from a more promising branch. This is what makes ToT useful for problems where one wrong intermediate step is fatal: Game of 24, crosswords, multi-step planning, constrained creative writing. The cost is high. Each tree node is a model call. A small tree (branching factor 3, depth 4) is dozens of calls instead of one.

**Tree of Thoughts** (ToT) prompting explores multiple reasoning branches at each step rather than committing to one linear chain. The model generates several candidate next thoughts at each node, an evaluator scores each one, weak branches are pruned, and the search continues from the strongest. The pattern was introduced by Yao et al. in 2023 (arXiv 2305.10601). The structural difference from chain-of-thought is the shape of the reasoning. Chain-of-thought is a sequence: one thought after another, with no branching. ToT is a tree: at each step, several next thoughts are generated, scored, and pruned. The model serves as both the expansion engine (generating thoughts) and the value function (scoring them via self-evaluation). The problem ToT solves is the irreversibility of linear reasoning. Chain-of-thought commits to one path. If an intermediate step is wrong, every later step inherits the error. ToT lets the search backtrack: a low-scoring branch is pruned and the search continues from a more promising one. This matters most on problems where wrong intermediate steps cannot be recovered from with linear reasoning. Puzzles like Game of 24 (find an arithmetic expression using four numbers that equals 24), crosswords, multi-step planning, and constrained creative writing are the canonical wins. The original paper showed dramatic accuracy gains on these task families. The cost is high. Each tree node requires a model call, and the total cost scales with branching factor times depth. A typical ToT run is 10 to 100x the cost of plain chain-of-thought. The pattern is used selectively where accuracy on hard search-shaped problems matters more than cost.

Quick reference

TL;DR

Tree of Thoughts explores several reasoning branches at each step, scores them, prunes weak ones, and continues from the strongest. It lets the agent backtrack, which plain chain-of-thought cannot.

Memory aid

Easy to grasp

Chain-of-thought is like walking through a maze and writing down your path as you go. If you take a wrong turn, you cannot undo it; your reasoning just continues from the bad branch. Tree of Thoughts is like standing at each junction in the maze, imagining what each direction would look like a few steps ahead, ranking which looks best, and only then committing to a direction. You can also back up if a direction turns out to be bad. The agent thinks of several next steps at each point, scores them, picks the best, and continues. It costs more because you generate and grade many possibilities instead of just one, but on hard puzzles where one wrong step ruins the whole answer, it works much better.

Key concepts

Tree of Thoughts Chain-of-thought Search Self-evaluation

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

Define ToT as a search tree where the model is both expansion engine and value function. Contrast with chain-of-thought (greedy depth-first) and self-consistency (N independent rollouts plus voting). Name the problem class (search-shaped with locally detectable failure). Attribute to Yao et al. 2023. Close with the cost reality and how the pattern's ideas now live inside frontier reasoning-mode models.

Real-world examples

Real products, models, and research that use this idea.

The original ToT paper (Yao et al. 2023) showed plain CoT reaches 4 percent on Game of 24, while ToT reaches 74 percent on the same task with GPT-4.
Frontier reasoning models (OpenAI o3, Claude Opus 4.7 extended-thinking, Gemini 3.1 deep-think) use search-like internal reasoning loops that resemble ToT, allocating hidden tokens to promising branches.
LangChain and LangGraph both ship ToT example implementations in their docs as reference patterns for puzzle-style tasks; they are rarely used in production due to cost.
Graph of Thoughts (Besta et al. 2023) extends ToT by allowing thoughts to be combined or revisited across branches, useful for tasks where partial solutions can be merged.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does ToT differ from self-consistency CoT?

QWhy is the self-evaluator the load-bearing component of ToT?

QWhere does ToT live in 2026 production?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Describing ToT as just running chain-of-thought multiple times and voting on the answer. That is self-consistency. ToT branches at every intermediate step, scores intermediate states, and prunes.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.

Yao et al., Tree of Thoughts: Deliberate Problem Solving with Large Language Models