Catastrophic Forgetting

Walk through the triage order when a customer support LoRA tanks MMLU by 4 points post-deploy

How much general data should a replay buffer for domain SFT use, and from which general dataset?

How should domain and general data balance in a code assistant SFT mix?

How do pretraining and fine-tuning differ in data and in the objective?

Which of these belong on the ship checklist for a production fine-tuned model?

Why does 'fine-tune on our company docs' usually fail, and what should you do instead?

What is the best architecture for a chatbot over a frequently updated company knowledge base?

Match training regime to sensible peak LR range

Why does fine-tuning on company docs often raise hallucination instead of lowering it?

How do you design the post-FT eval suite for a customer support fine-tuned model?

Which buckets MUST a post-FT eval suite include for a customer support fine-tuned model?

Which of these reduce catastrophic forgetting?

PremiumHow do you detect…

Which signal is the most reliable indicator of catastrophic forgetting?