**Prompt rot** is when a prompt that used to work well in production starts producing worse outputs over time, even though nobody changed the prompt itself. The code is the same, but the answers get worse.
There are three usual culprits. First, the model behind the API changes. OpenAI, Anthropic, and Google all ship updates that shift behavior in subtle ways. Second, the data flowing into the prompt shifts. If you do RAG, the documents in your index change as you ingest new content, and the prompt's assumptions about what context looks like quietly stop holding. Third, your users start asking new things. A prompt designed for refund questions starts seeing returns, shipping, and gift-card questions.
None of these show up in a unit test. The only way to catch rot is to run a regression eval on a representative golden set, on a schedule, and watch the curves.
**Prompt rot** is the silent quality degradation of a production prompt without any code change. The file is identical to what shipped on day one, yet output quality slides over weeks or months.
Three drivers cause it. Model version drift: the provider pushes a quiet upgrade and behavior shifts, especially around format strictness and refusal thresholds. Retrieval-distribution drift: the documents your RAG pipeline pulls in change as you ingest new content, so the chunks the prompt sees no longer match what it was tuned for. Usage drift: users discover the product and start asking questions the prompt was never built for, expanding the input distribution.
The failure mode is that none of this is visible without continuous evaluation. Unit tests pass because they pin specific inputs. The fix is regression CI on a representative golden set of 100 to 500 cases. Bootstrap CIs on the metrics, and sample production traces into a recent-failures bucket that feeds back into the eval set. Run on every prompt change and on a schedule, so model-side drift and data-side drift both trigger alerts before users notice.
Quick reference
TL;DR
Prompt rot is the silent quality decay of a production prompt whose code never changed, driven by model upgrades, retrieval drift, and new user behaviors the prompt was not built for.
Imagine writing a recipe for a friend who cooks for you every weekend. The recipe stays the same on paper, but your friend slowly switches brands, the grocery store rearranges its shelves, and you start inviting guests with new diets. Nothing on the page is wrong, yet dinner gets worse week by week. Production prompts decay the same way. The model behind the API gets a quiet upgrade, the documents flowing in shift toward new topics, and users start asking things the prompt was never designed to handle. Nobody touched the prompt, but output quality slides.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Internal prototypes with a fixed test fixture and no real user traffic There is no drift surface to monitor; a static eval is enough until the prototype ships to real users.
Stateless tool calls with hard schema validation on the output Schema validation catches format regressions immediately; the broader regression eval is overkill when the only failure mode is parse error.
One-off batch jobs against a frozen model snapshot and a frozen dataset Both axes that drive rot are pinned, so the run is reproducible by construction.
Quick scan · 2–4 min
Everything important, quickly.
3 min: the three drift axes + regression-CI mitigation + version pinning + golden-set refresh cadence + production-trace feedback loop.
Real-world examples
Real products, models, and research that use this idea.
1Teams using Claude Opus 4.7 via the Anthropic API pin a dated snapshot in production and re-baseline their golden eval whenever they roll forward, because behavior on format strictness shifts between snapshots.
2Perplexity continuously re-evaluates synthesis prompts against fresh web retrievals, since the source distribution shifts faster than any static eval set could track.
3GitHub Copilot Chat samples production traces into a regression bucket, then weights recent failures into the next golden-set refresh.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you detect prompt rot before users complain?
Sign in or go Pro to see the approach for this follow-up.
QWhat changes if you switch from a pinned model to a floating one?
Sign in or go Pro to see the approach for this follow-up.
QHow does retrieval drift in RAG differ from model drift?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Assuming a prompt that worked at launch will keep working forever, because no code changed and tests still pass on the original golden cases from a year ago.