Why gate every prompt or model change behind a held-out eval in CI?

Attempt it

Explain why an LLM product should run a held-out eval set on every prompt or model change in CI, and what failure this prevents.

Free · 2 AI evals / day

Quick reference

TL;DR

Prompt and model changes regress cases you didn't look at, invisibly. A held-out eval in CI turns quality into a blocking number that catches drift before users do.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine tuning a recipe to make one picky friend happy. You add more salt, they love it — but you never check whether everyone else now finds it too salty. A held-out eval set is like keeping a panel of taste-testers who try every version of the recipe. Before you serve the new version to guests, the panel scores it. If the score drops, you don't serve it. That way one fix for one friend can't quietly ruin the dish for the whole table — and 'I think it's better' becomes an actual score you can trust.

Key concepts

LLM Evaluation LLM System Design

Quick scan · 2–4 min

Everything important, quickly.

Spend 5-6 minutes: open with the missing-compiler analogy, explain non-local effects and invisible drift, then the blocking gate and the coverage caveat.

Side-by-side

Without eval-in-CI	With eval-in-CI
'This prompt feels better'	Aggregate score on representative cases
Regressions found by users in prod	Regressions fail the build pre-release
Silent model updates go unnoticed	Scheduled run flags the drift
Fix one case, break unknown others	Per-case diff shows what regressed

Real-world examples

Real products, models, and research that use this idea.

promptfoo running an eval suite as a CI step that fails the build when a prompt change drops the pass rate.
LangSmith / Braintrust gating prompt and model changes behind dataset scores before merge.
Teams catching a silent provider model update via a scheduled nightly eval that flags an unexplained score drop.
Seeding the held-out set from real thumbs-down production cases so the gate covers actual failure modes.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow do you keep your held-out eval set from going stale as the product evolves?

Sign in or go Pro to see the approach for this follow-up.

QYour eval is green but users still report worse answers — what's likely wrong?

Sign in or go Pro to see the approach for this follow-up.

QHow do you score open-ended outputs where there's no exact-match answer?

Sign in or go Pro to see the approach for this follow-up.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.

Your AI Product Needs Evals (Hamel Husain)