Membership inference on fine-tuned models. Why are small FT datasets vulnerable?

A **membership inference attack** tries to answer one question: was this specific example part of the model's fine-tuning data? The attacker does not need to see inside the model. They only watch how it behaves. The trick relies on memorisation. A model assigns lower loss, meaning higher confidence, to text it was trained on. The attacker measures the model's loss on a candidate example. They compare it to the loss on similar text known to be outside the set. A big gap suggests the example was a training member. Small fine-tuning sets make this easy. With only a thousand examples, the model revisits each one many times during training. It often memorises them word for word. The loss on those memorised examples drops far below normal text. That matters because the data leaks. Prompt the model with the start of a memorised record and it may complete the rest verbatim, including names, emails, or private details. Mitigations include scrubbing personal data before training, training gently with fewer passes, deduplicating the data, and using differential privacy for formal guarantees.

A **membership inference attack** decides whether a particular example was in the fine-tuning set, using only query access to the model. The signal is loss. Trained-on examples receive lower loss and higher per-token probability than unseen text. The attacker scores a candidate example, then compares it against a reference distribution of out of set text. A statistically significant gap flags membership. Stronger variants calibrate per-example difficulty using a shadow model or a likelihood ratio, so easy and hard examples are judged fairly. Small datasets are the core vulnerability. Each example carries a large share of the total gradient signal. A thousand-example set gives each record roughly a thousand times more influence than a record in a billion-token corpus. The model memorises some examples verbatim, so their loss collapses. The visible symptom is completion. Prompt with a unique prefix and the model reproduces the rest, leaking any embedded private data. Mitigations stack by strength. Redact personal data before training. Deduplicate aggressively. Lower the learning rate and run fewer epochs. For formal protection, train with DP-SGD, which bounds any single record's influence at a measurable utility cost. Regulated teams typically combine redaction, LoRA, and DP-SGD.