Data Leakage

A small medical fine-tune leaks verbatim credit-card snippets under red-teaming: identify the failure mode and the standard mitigation

Spot the bug: packed-sequence SFT where the model learns to copy across unrelated examples

Why does using the same model family as synthetic-data teacher AND eval judge inflate FT scores?

When eval prompts appear verbatim inside an SFT file after a synthetic-data run, what's the root cause?

A model aces a benchmark with suspiciously high scores. The first hypothesis is contamination. Explain what that means to a non-technical PM.

Spot the flaw: 'We fine-tuned on 10K examples and eval on a random 500 from the same dataset. Accuracy is 94%, so the model is ready for production.'

Non-obvious modes of train/test leakage in instruction-tuning evaluation

Which of these are real train/test leakage modes for an instruction-tuning project?