Which of these reduce catastrophic forgetting in practice?

**Catastrophic forgetting** is when a model gets good at your new fine-tuning task but loses general skills it had before. Fine-tuning drags the weights toward your data, and if you drag too hard, the old behavior breaks. The correct answers all share one theme: move the model less, or move only a small part of it. A lower learning rate with fewer epochs takes gentler steps, so the model never overshoots the abilities it relied on. A parameter-efficient method like LoRA or QLoRA freezes the original weights entirely and learns a small adapter on top. The base brain is untouched, so it keeps its general capability by default. Mixing in a small slice of general data, called a replay buffer, keeps the model practicing old skills while it learns the new ones. This balances the gradient so it does not drift too far in one direction. The wrong answers do the opposite or nothing. Bigger batches push harder, more epochs train longer on the narrow domain, and switching the loss to mean-squared-error simply breaks language model training.

**Catastrophic forgetting** happens when fine-tuning pulls weights so far from their pretrained values that general capability degrades. Every real mitigation reduces how far the weights move. Three of the options do exactly that. A lower learning rate with fewer epochs shrinks each step, so the model never overshoots the directions it relied on for general skill. A frozen-base method like LoRA or QLoRA leaves the original weights untouched and learns a small rank-limited adapter, so pretrained behavior survives by default. A replay buffer, mixing 5 to 15 percent general data into the fine-tuning set, keeps the gradient balanced and stops one-sided drift. The distractors fail for clear reasons. A larger batch size sharpens the gradient signal and can move the model more decisively, which speeds up forgetting unless you drop the learning rate too. Swapping cross-entropy for mean-squared-error breaks training, since language modeling needs a probabilistic target. Training for many more epochs is the opposite of the fix, more time on the narrow domain makes forgetting worse. The unifying test is simple. Anything that limits gradient drag helps; anything that intensifies training on the narrow task hurts.