What qualifies as benchmark contamination in pretraining?
Contamination is when a benchmark's test items, their answers, or close variants appear in training data, so the score can measure recall instead of the skill the benchmark tests.
A teacher writes a final exam and keeps it locked away. If a copy leaks and a student studies it, the student's score goes up, but it no longer shows how well the student knows the subject. It shows that the student saw the exam. The leak still counts if the copy was reworded, and it is worst if the answers were written in. Not every overlap is a leak. If the exam asks questions about a famous poem and the student has read the poem before, the student still has to work out the answers. For a language model, the exam is a benchmark and the study material is the pretraining data. Benchmark contamination means the test items, their answers, or close rewordings of them got into that data.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
A benchmark score is meant to estimate how well a model handles new examples of a task. That estimate holds only if the test items are new to the model. Benchmark contamination is the case where they are not: test items, their answers, or close variants of them appeared in the training data.
Web-scale pretraining makes this easy to cause by accident. Benchmarks are published on GitHub, quoted in papers and discussed on forums, and those pages end up in crawls. The work has three parts: define what counts as a leak, measure how much leaked, and test whether it changed the score.
What counts as contamination
Contamination is defined by the evaluation items, not by the topic. A model that learned arithmetic from millions of math web pages and then scores well on a math benchmark has generalized. A model that saw the benchmark's own problems has not been tested fairly on them. Three kinds of overlap count:
- Test inputs. The question, passage or prompt of a test item appears in the training data. This gives partial help: the model may recognize the item and recall related text.
- Inputs with labels. The question appears together with its answer, for example in a GitHub copy of the dataset or a forum post that solves it. This is the most damaging case, because the model can memorize the exact mapping from item to answer.
- Close variants. The same item reformatted, translated, reworded, or with names and numbers changed. A variant teaches the same item-specific answer, so it counts, even though string matching often misses it.
Validation and development splits count as well when they are used to report results or tune choices. So do benchmark solutions published after the benchmark, which keep spreading long after the original release.
Contamination is a relation between one model's training data and one benchmark. The same corpus can be clean for a benchmark created after it was crawled and contaminated for an older one. A new crawl of the same web can contaminate a benchmark that an earlier crawl did not.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- GPT-3: a filtering bug left some benchmark overlaps in the training data, so the team reported clean-subset scores and marked its PIQA and Winograd results with an asterisk.
- Llama 2 ran a token-level contamination analysis and found evidence of a score boost only on HellaSwag and MMLU-Humanities.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy is a 13-gram match rule both too strict and too loose?
It flags shared passages and boilerplate that give away no answers (false positives) and misses reworded or translated items (false negatives).
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Checking only for exact string matches. Reformatted, translated or paraphrased test items slip past n-gram filters and can still inflate scores.
60 second bullets to scan on the way to the call.
Definition of benchmark contamination
Input-only leakage against input plus label leakage
Primary sources. Browse if you want the original framing.
- Brown et al., Language Models are Few-Shot Learners
- Touvron et al., Llama 2: Open Foundation and Fine-Tuned Chat Models
- Yang et al., Rethinking Benchmark and Contamination for Language Models with Rephrased Samples
- Zhang et al., A Careful Examination of Large Language Model Performance on Grade School Arithmetic
- Srivastava et al., Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Same topic, related formats. Practice these next.