Tree of Thoughts explores multiple reasoning branches at each step, scores them, prunes the weak ones, and continues from the strongest, instead of committing to a single chain like chain-of-thought.
Imagine you are solving a maze with a pencil. Chain-of-thought is like walking forward one step at a time and never looking back, even when the path dead-ends. Tree of Thoughts is the way a real person solves a maze. At each junction you look at two or three possible paths, you guess which one seems most promising, you try it, and if it does not work you back up and try another. Tree of Thoughts asks the language model to do exactly that. At every step it generates several candidate next thoughts, rates them, keeps the best, drops the rest, and moves on. The shape of the reasoning is a tree, not a straight line, which is why the name fits.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Tree of Thoughts is one of the more memorable prompting techniques from the 2023 wave, and it sits in an awkward place in 2026. It still wins on the problems it was designed for, but the rise of reasoning-mode models has shifted the cost-benefit calculus in ways every senior interview will probe.
The right way to think about ToT is as a search algorithm running on top of an LLM. The LLM plays two roles: a generator that proposes candidate next thoughts at each step, and an evaluator that scores them so the system can prune. The reasoning structure stops being a single chain and becomes a tree; the algorithm wrapping it is just BFS or DFS with pruning.
This deep dive defines ToT precisely, contrasts it with chain-of-thought and self-consistency, walks through the call-count cost that determines when it is worth using, and explains how reasoning-mode models change the picture.
From chain to tree: what ToT actually changes
Chain-of-thought prompting asks the model to produce one reasoning chain (a sequence of intermediate steps) and then an answer. The chain is generated greedily: the model commits to each step as it writes it, and there is no mechanism to reconsider. If the first step is wrong (a wrong subgoal, a wrong arithmetic move, a wrong intermediate claim), every subsequent step inherits the error and the final answer is usually wrong.
Tree of Thoughts removes the commit-once constraint. At each reasoning step the system asks the model to generate multiple candidate next thoughts (say, 3-5 branches). Each branch is then scored by an evaluator (the same LLM asked to rate how promising the branch looks, on a scale or a categorical label). The branches with the lowest scores are pruned; the survivors become the parents of the next round of branching. The full process traces out a search tree, not a chain.
The wrapping algorithm can be BFS (expand all surviving branches one level at a time), DFS (go deep on the most promising branch first), or beam search (keep the top-k survivors at each level). The original Yao et al. 2023 paper describes both BFS and DFS variants; in practice BFS with beam-search style pruning is the most common shape.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Yao et al. 2023 reported ToT solving 74% of Game of 24 problems vs CoT's 4% with GPT-4, illustrating the gap on backtracking-heavy puzzles.
- OpenAI's o-series and Anthropic's extended-thinking modes (Claude Opus 4.7) internalize backtracking-style reasoning, reducing the marginal benefit of explicit ToT scaffolding.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does Tree of Thoughts differ from self-consistency, which also uses multiple reasoning paths?
Self-consistency samples N complete chains and majority-votes the final answer. ToT builds a tree with pruning at every step. Different algorithmic shape and different cost profile.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating Tree of Thoughts as a small upgrade over chain-of-thought you can drop into any prompt. ToT is a search procedure with many LLM calls per question, not a single prompt template, and it is overkill for tasks that do not need backtracking.
60 second bullets to scan on the way to the call.
Define ToT in one sentence; the tree versus chain framing
What the evaluator does and why it is essential
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.