What does counterfactual data augmentation measure in LLM bias evaluation?
Counterfactual data augmentation swaps one demographic attribute in an otherwise identical prompt and checks whether the model's output changes. A change signals demographic bias.
Imagine you write the exact same loan-application story twice. In one copy the applicant is named John; in the other she is named Aisha. Everything else, income, job, credit history, stays word for word identical. You feed both to the model and compare what it says. If the model approves John but hesitates on Aisha, the only thing that changed was the name, so the difference must come from the name itself. That is bias. Counterfactual augmentation is just this trick scaled up: take many prompts, flip one demographic detail (name, pronoun, ethnicity) at a time, and measure how often the answer flips with it. Identical answers across the swap mean the model treated the people the same.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Counterfactual data augmentation (CDA) is one of the cleanest ideas in fairness evaluation, and that cleanliness is exactly why it shows up in interviews. The premise is causal: if you change one demographic attribute in a prompt and hold everything else perfectly fixed, then any change in the model's output must be caused by that attribute. No correlation hand-waving, no confounders to argue about, at least in the idealised case. This is what makes it more rigorous than simply slicing aggregate metrics by group, where unobserved differences between groups can masquerade as bias or hide it.
This question tests whether you know what CDA measures as an evaluation, and whether you can distinguish it from the identically-named training-time technique. The correct answer (option A) frames CDA as a measurement of output invariance under attribute swaps. The two distractors that are not the training technique, options C and D, describe lexical diversity and confidence-score comparison; neither involves swapping an attribute while holding the scenario fixed, so neither is the counterfactual method.
The deep dive below walks the construction, the invariance logic, the eval-versus-training disambiguation that trips up mid-level candidates, the confounds that a senior engineer controls for, and where the technique sits inside a real production bias suite.
The minimal pair: change one thing, fix the rest
The atom of counterfactual evaluation is the minimal pair: two prompts that are byte for byte identical except for one protected attribute. "Dear hiring manager, John has five years of backend experience and asks about a senior role" versus the same sentence with "Aisha" in place of "John." The job, the tenure, the wording all stay frozen. Only the name moves.
The reason this works is counterfactual reasoning. The word counterfactual means "contrary to fact", that is, what the output would have been had one variable taken a different value. Because the pair differs on exactly one variable, the difference in output is the causal effect of that variable. Everything else is held constant by construction, so there is nothing else to blame.
You then scale this from one pair to thousands, sweeping names, pronouns, and stated group memberships across a fixed library of scenarios. The aggregate of those per-pair differences becomes your bias metric.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- BBQ (Bias Benchmark for QA) uses templated counterfactual pairs to measure how demographic context shifts model answers under ambiguity.
- WinoBias swaps gendered pronouns in coreference templates to measure whether models lean on occupational gender stereotypes.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you design a name-swap probe so the gap is attributable to a single demographic axis?
Hold the scenario fixed and vary one axis at a time using a controlled name list matched on length, frequency, and tokenisation. Swap gender separately from ethnicity, and report a per-axis divergence rather than a single blended name swap.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Confusing the eval-time and train-time meanings of the term. As an evaluation, counterfactual augmentation measures bias by swapping attributes. As a training trick, it instead reduces bias by adding swapped examples.
60 second bullets to scan on the way to the call.
What the word counterfactual means and what stays fixed in a pair
How a minimal pair isolates one protected attribute
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.