Describe the Reflexion mechanism (Shinn et al. 2023). How does it allow an agent to improve on retries without gradient updates, and what is the key data structure it relies on?
Free · 2 AI evals / day
**Reflexion** is a way to let an agent get better across retries without ever changing the model's weights. Normally a model improves through fine-tuning, which means running gradient updates on its parameters. Reflexion skips all of that.
The trick is verbal self-reflection. After an attempt fails, the agent writes a short, plain-language critique of what went wrong and why. For example, it might note that it called the wrong tool or misread an error message. That critique is the lesson learned.
The critique is saved in an episodic memory buffer, which is just a growing list of past attempts and their critiques. It lives outside the model, like a notebook the agent keeps.
On the next try, the stored critique is pasted into the top of the new prompt. The model reads its own past advice before acting again, so it can avoid repeating the mistake. Because the lesson sits in the prompt and not in the weights, no fine-tuning happens.
This is in-context learning: the model adapts purely from what is in its context window, not from a parameter update.
**Reflexion** lets an agent improve across retries without any gradient update. After a trial fails, the agent generates a natural-language critique of what went wrong and why. That verbal self-reflection is the core mechanism.
The critique is written to an episodic memory buffer, a growing list of past attempts paired with their critiques. The buffer lives outside the model, in the runtime, not in the weights.
On the next attempt, the stored critiques are prepended to the prompt. The model reads its own prior lessons before acting, so the next trajectory is shaped by what failed last time. The loop is reflect, store, retry.
Because the lesson sits in the context window rather than the parameters, no fine-tuning or backpropagation is needed. This is in-context learning acting as the functional substitute for a gradient step.
One hard requirement is a usable feedback signal. Reflexion needs to know a trial failed, from a unit test, a task reward, or an evaluator model. Without that signal it cannot write a meaningful critique. The buffer also grows with each trial, so old entries are summarised or truncated to fit the context window.
Quick reference
TL;DR
Reflexion turns a failed attempt into a written self-critique, stores it in episodic memory, and prepends it to the next try, so the agent learns in-context with no weight update.
Imagine a student who fails a practice exam. Instead of being reprogrammed, they write a short note to themselves: 'I lost marks because I rushed the algebra and skipped checking units.' Before the next attempt they read that note. They have not gotten smarter in any permanent way, but with the note in front of them they avoid the same mistakes and do better. Reflexion works like this for an agent. After a failed run, the agent writes a plain-language critique of what went wrong. That note is saved in a little notebook called episodic memory. On the retry, the note is pasted at the top of the prompt so the model reads its own advice before trying again. Nothing inside the model changes. The only thing that changed is what it now remembers about its last mistake.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
There is no reliable way to tell whether a trial succeeded or failed. Reflexion needs a reward, test, or evaluator. With no signal the critique is uninformed and often wrong.
The task must be solved correctly on the first attempt with no retries allowed. Reflexion only helps across multiple trials. A single-shot constraint removes the loop it depends on.
You need a durable capability gain that survives across sessions and fresh contexts. The lesson lives in the prompt, so it disappears once the buffer is cleared. Fine-tuning is the durable option.
Latency or cost per task is already at its ceiling. Each extra trial adds a full critique and retry round trip, and the growing buffer inflates token spend.
Quick scan · 2–4 min
Everything important, quickly.
Define Reflexion as verbal self-critique stored in episodic memory and prepended to the next prompt, frame the buffer as an in-context learning signal that substitutes for a gradient step, then close with the limits: it needs a reward signal, learns nothing permanent, and can reinforce a wrong lesson.
Real-world examples
Real products, models, and research that use this idea.
1Coding agents like Cursor and Cline run a Reflexion-style loop: a failing test suite is the reward signal, and the agent writes a note about the failure before its next edit attempt.
2LangGraph ships reflection patterns where an evaluator node critiques a draft and the critique is fed back into the generator node on the next loop iteration.
3The original Reflexion paper reported large gains on HumanEval coding and ALFWorld decision tasks purely by adding the verbal self-critique buffer.
4Modern frontier agents built on Claude Opus 4.7 or GPT-5.5 use evaluator-driven self-critique on SWE-bench style tasks, retrying failed patches with the prior critique in context.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does Reflexion differ mechanically from reinforcement learning from human feedback?
Sign in or go Pro to see the approach for this follow-up.
QHow would you stop Reflexion from reinforcing a wrong lesson across trials?
Sign in or go Pro to see the approach for this follow-up.
QHow do you keep the episodic buffer from overflowing the context window on long retry chains?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Claiming Reflexion fine-tunes the model. It never touches the weights. The lesson lives in the prompt, so improvement comes from in-context learning, not a gradient step.