What does counterfactual data augmentation measure in LLM bias evaluation?
Counterfactual data augmentation swaps one demographic attribute in an otherwise identical prompt and checks whether the model's output changes. A change signals demographic bias.
Imagine you write the exact same loan-application story twice. In one copy the applicant is named John; in the other she is named Aisha. Everything else, income, job, credit history, stays word for word identical. You feed both to the model and compare what it says. If the model approves John but hesitates on Aisha, the only thing that changed was the name, so the difference must come from the name itself. That is bias. Counterfactual augmentation is just this trick scaled up: take many prompts, flip one demographic detail (name, pronoun, ethnicity) at a time, and measure how often the answer flips with it. Identical answers across the swap mean the model treated the people the same.
Detailed answer & concept explanation~6 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
5 min: define the counterfactual minimal pair, the invariance logic, the eval versus training disambiguation, confounding and intersectional pitfalls, and where it fits in a production bias suite.
Real products, models, and research that use this idea.
- BBQ (Bias Benchmark for QA) uses templated counterfactual pairs to measure how demographic context shifts model answers under ambiguity.
- WinoBias swaps gendered pronouns in coreference templates to measure whether models lean on occupational gender stereotypes.
- Anthropic and OpenAI bias evals include name-swap and pronoun-swap probes as part of pre-release model cards in 2026.
- Patronus and Galileo ship counterfactual fairness checks as configurable CI gates over a team's own query distribution.
- HuggingFace Evaluate exposes regard and honest metrics that compare generated text across swapped demographic groups.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you design a name-swap probe so the gap is attributable to a single demographic axis?
QWhy can a model pass single-attribute counterfactual probes yet still be intersectionally biased?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Confusing the eval-time and train-time meanings of the term. As an evaluation, counterfactual augmentation measures bias by swapping attributes. As a training trick, it instead reduces bias by adding swapped examples.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.