Only 200 examples for a niche format task: which PEFT method gives the best shot?

The right answer is C. With only 200 examples, the danger is not that the model is too weak to learn the format. The danger is that it learns the 200 rows by heart and falls apart on the 201st example. The number of trainable parameters acts like a memory budget. Give the model millions of trainable weights and it has more than enough room to memorize 200 short patterns one by one. Give it only a few tens of thousands, and it cannot afford to memorize; it has to compress what it sees into a few general rules. IA3 learns just a handful of scaling vectors per layer. Small-rank LoRA at r=2 or r=4 learns two thin matrices per target weight. Both expose a tiny trainable surface, which works as a built-in regularizer in this low-data regime. Full fine-tuning and r=64 LoRA are the opposite mistake: they hand the model a huge memory budget and almost guarantee overfitting within a few hundred steps.

**Option C** is the right call. The binding constraint on a 200-example dataset is overfitting risk, not capacity. The PEFT method you pick should expose as small a trainable surface as still captures the format. IA3 learns three scaling vectors per attention plus MLP block, typically tens of thousands of trainable parameters across a 7B base. Small-rank LoRA at r=2 or r=4 is a similar order of magnitude. Both act as strong implicit regularizers: the model literally cannot allocate enough capacity to memorize 200 rows, so it is forced to compress what it sees into a few general directions. Full fine-tuning is the worst option here because every weight gets a gradient; on 200 examples it overfits within a few hundred steps and the eval loss curves up while train loss keeps dropping. LoRA at r=64 across all-linear targets pushes trainable parameters into the tens of millions; same failure mode, just slower to show. Prompt tuning sometimes wins in few-shot classification, but it is fragile on format-heavy niche tasks and rarely the default in 2026 practice. The reliable low-data recipe is small adapters, short training, and aggressive early stopping.