What is the Reflexion mechanism and how does it avoid fine-tuning?
Reflexion replaces gradient updates with verbal self-critiques: after a failed trial the agent writes what went wrong, stores it in memory, and prepends it to the next attempt.
Imagine a student who keeps failing the same kind of exam question. They never get smarter on the spot, but after each try they scribble a sticky note: 'I forgot to check the units last time.' Before the next attempt they read all their sticky notes, so they avoid repeating those mistakes. The student's brain has not changed at all. Their notes have. Reflexion works like this for an agent. The model itself is frozen, so its weights never move. After a failed attempt it writes a short note in plain language explaining the slip-up. That note is saved, then pasted into the prompt for the next attempt. The agent improves trial after trial purely because it now reads its own past lessons, not because it was retrained.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Reflexion is a technique for letting a language agent improve over repeated attempts at a task without ever touching the model's weights. The headline idea is to replace the gradient update of classical learning with a critique written in plain language. After a failed trial, the agent reflects on what went wrong, records that reflection in memory, and reads it back before the next attempt.
This is why the method is described as verbal reinforcement learning. The reinforcement signal still exists, but it is carried as text in the prompt rather than as a change in parameters. The model is a frozen black box from start to finish. All the learning lives outside it, in an episodic memory buffer that the runtime maintains across trials.
The distinction matters because most ways of making a model better at a task assume you can train it. Reflexion targets the common production reality where the model is a hosted endpoint you cannot fine-tune, the task is narrow, and you have only a handful of attempts to get it right. It turns the model's own ability to describe its mistakes into the optimisation loop, which is a surprisingly powerful substitute for backpropagation on the right kinds of problems.
The three roles in the loop
Reflexion decomposes a retry loop into three cooperating roles, all played by language models or simple checks.
The first is the actor. It attempts the task and produces a trajectory, which is the full sequence of actions, tool calls, and outputs for one trial. The actor is the part that would, in a normal agent, simply try once and stop. In Reflexion it is wrapped so that its output can be judged and reflected on before it runs again.
The second is the evaluator. It inspects the finished trajectory and produces a signal that says whether the trial succeeded or failed. This signal can come from unit tests on generated code, an exact-match check against a known answer, a reward function, or another model acting as a judge. The evaluator does not need to be a model at all. For a coding task it is often just the test runner, which is why Reflexion shines on problems with a cheap, objective check.
The third is the self-reflection model. It reads the trajectory together with the evaluator's signal and writes a short critique. The critique is not a fix applied to the code or the answer. It is a lesson in words, such as a note that the agent skipped a boundary case or misread an error message. The same underlying model usually plays both the actor and the self-reflection role, just with different prompts, so no extra checkpoint is required to run the loop.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Coding agents like Cursor and Aider follow a Reflexion-style retry loop: run the test suite, read the failure, write a note on the bug, and edit again with that note in context.
- A frontier model such as Claude Opus 4.7 or GPT-5.5 driving a SWE-bench agent can retry a failed patch after reflecting on the failing test output, without any weight change.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does Reflexion differ from a policy gradient method like PPO at the level of the update?
Contrast the credit-assignment channel. PPO maps a scalar reward to a parameter update via backprop. Reflexion maps the outcome to a natural language critique stored in memory and consumed through the prompt, leaving parameters fixed.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Saying Reflexion fine-tunes the model on its mistakes. It never touches weights. The self-critique lives in the prompt as in-context feedback, not in a gradient step.
60 second bullets to scan on the way to the call.
State that Reflexion performs no gradient update and keeps weights frozen.
Name the three roles: actor, evaluator, and self-reflection.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.