Spot what this continual pretraining plan gets wrong about forgetting

Spot the error·Medium·4.0 · 0·~2 min·Asked atGoldman Sachs Gong Hebbia

Attempt it

Click any words you think contain an error. Click again to unmark.

Mark at least one word to submit.

**Continual pretraining** is when you take a base model that already knows general language and keep training it on a new domain. The dream is that you add the new knowledge without losing the old. The reality is **catastrophic forgetting**: every gradient update moves all the weights, including the ones that store general capabilities. This plan has three concrete mistakes. First, training on pure PubMed text with zero general web data is the textbook recipe for forgetting. The standard mitigation is **replay**: keep a meaningful fraction (commonly 20 to 50 percent) of general-distribution tokens in the mixture so the model keeps practicing what it already knows. Second, warming up to the original peak LR of 3e-4 is aggressive on a shifted distribution. Continual pretraining uses a reduced peak (often 1/3 to 1/10 of the original), with a fresh warmup and decay schedule. High LR on a new distribution maximizes both the speed of forgetting and the risk of instability. Third, the claim that general capabilities are frozen into the weights is false. Nothing is frozen. Every parameter update reshapes the same weights that hold MMLU performance, coding ability, and reasoning. Without replay and a conservative LR, those scores will degrade measurably.

Continual pretraining is a distribution shift, and distribution shifts cause forgetting unless you actively counter them. The plan ignores all three standard countermeasures. - **No replay data.** Pure PubMed style text with zero general web data trains the model to model the new distribution at the cost of forgetting the old one. The fix is to blend in 20 to 50 percent general-distribution replay tokens (often a mix of the original pretraining sources) so the model continues to practice the capabilities you care about preserving. - **Original peak LR.** Re-warming to 3e-4 is the LR that was right for a fresh run on a stable distribution. On a shifted distribution, it produces faster adaptation but also faster forgetting and a higher chance of instability. Continual pretraining practice uses a substantially lower peak (typically 1/3 to 1/10 of the original) with a fresh warmup and decay. - **Frozen-weights claim.** Gradient descent moves all parameters every step. Nothing about the base pretraining is locked in. Without replay and a conservative LR, expect single-digit to double-digit point drops on MMLU and coding benchmarks; treat those as primary regression gates, not as assumed-stable outputs. The correct plan combines a mixed corpus (PubMed plus general replay), a reduced peak LR with fresh schedule, and a regression-gate eval suite that monitors both target-domain gains and general-capability retention. If general scores fall below tolerance, you reduce the LR or raise the replay fraction.

TL;DR

Continual pretraining without replay, at the original peak LR, and without general-capability regression gates is the textbook recipe for catastrophic forgetting.

Memory aid

Easy to grasp

Picture a doctor who decides to spend a year reading nothing but cardiology papers. By the end she will know cardiology much better, but she will also have forgotten chunks of dermatology, pediatrics, and pharmacology because she was not practicing them. To stay sharp on the rest of medicine while specializing, she keeps a mix of general cases on her schedule. This plan instead tells the model to read only cardiology for the equivalent of a year, at full study intensity, and then claims the model will somehow remember everything else. It will not.

Key concepts

Pretraining

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Continual pretraining is one of the most useful and most over-promised techniques in the LLM toolbox. Used carefully, it adds domain depth to a strong base model. Used carelessly, it produces a model that is mediocre at the new domain and noticeably worse at everything else. The plan in this question is the careless version, and the failures it embeds are common enough that almost every team has shipped a version of them at least once.

This walkthrough develops the catastrophic-forgetting default, the three countermeasures, and the Pareto-frontier framing that lets you reason about the tradeoff explicitly rather than wishing it away.

Why forgetting is the default

Gradient descent does not respect prior training. When the model takes a step in parameter space, the step is computed from the current loss on the current batch. If the current batch is PubMed and the model was previously good at general web text, the gradient pushes the weights in a direction that improves PubMed loss. Whether that direction also degrades general capability depends entirely on whether the two loss surfaces are aligned, and in practice they often are not.

This is the textbook setup for catastrophic forgetting. The phenomenon is named, measured, and reproducible. A continual pretraining run with no countermeasures will degrade general benchmarks by single-digit to double-digit points depending on token budget, LR, and the size of the distribution shift.

The plan in this question commits to all three of the things that maximize forgetting: pure-domain data, original peak LR, no regression evals. The model will indeed get better at PubMed text. It will also get visibly worse at MMLU, HumanEval, and any reasoning benchmark.

Replay: the highest-leverage countermeasure

Learning rate: why the original peak is wrong

Regression gates and the Pareto frontier

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Meta's Llama 3 continual pretraining experiments use replay mixtures with tens of percent general data and reduced peak LR relative to the original base training run
BloombergGPT's domain adaptation work documents replay-mixture choices and reports both finance-domain gains and general-benchmark retention as paired metrics

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you pick the replay fraction for a new domain you have not adapted to before?

Run small proxy ablations at 0, 10, 30, 50 percent replay, measure domain gains and general regressions, fit the Pareto curve, and pick the point where general regression is within tolerance. Document the choice as part of the run plan.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Believing general capabilities are 'locked in' after base pretraining, so a domain-only continual run is safe, and then being surprised when MMLU and coding scores collapse.

60 second bullets to scan on the way to the call.

What is catastrophic forgetting and why is it the default in continual pretraining?
What replay fraction is typical and how is it chosen?

Primary sources. Browse if you want the original framing.