Pick the sensible `num_train_epochs` for a typical instruction-SFT run on a curated dataset

MCQ·Easy·4.0 · 0·~1 min

Attempt it

**Pretraining** runs over trillions of tokens and teaches the model language from scratch. **SFT** runs over a much smaller curated set, usually tens of thousands to a few hundred thousand examples, and only adjusts behavior toward instruction following and the desired output style. Because the base already knows the language, SFT does not need many passes. **1 to 3 epochs** is the standard range across nearly every published instruction-tuned model (Alpaca, Tulu, Llama, Mistral, Qwen, DeepSeek instruct variants). A single pass is often enough for large high-quality datasets. Two or three passes help when the set is smaller and the format adjustment needs reinforcement. Going higher than three epochs almost always overfits. Training loss keeps dropping, but the model memorises specific phrasings and held-out evaluation flattens or worsens. The output range narrows and becomes brittle on novel prompts. Fractional epochs (A) under-train; dozens (C) or fifty plus (D) are pretraining-scale numbers that do not fit a small curated SFT set.

**1 to 3 epochs** is the standard SFT range across nearly every published instruction-tuned model. The reasoning is about data scale and what SFT is actually doing. Pretraining sees trillions of tokens over many weeks or months; the model learns language, syntax, world knowledge, basic reasoning. SFT sees tens of thousands to a few hundred thousand curated examples; its job is to adjust the model's output distribution to follow instructions and produce responses in the desired style. Because the base model already knows the language, SFT does not need many passes. The typical published numbers are: - **1 epoch** for large curated SFT sets (100K plus examples), often the default in Tulu and Llama 3 recipes. - **2 to 3 epochs** for smaller SFT sets where the format adjustment needs more reinforcement. - **More than 3** is rarely justified; the dataset is being memorised rather than learned from. The diagnostic for overfitting is the gap between training loss and held-out evaluation. After about three epochs on a typical curated set, training loss continues dropping while held-out evaluation flattens or drops. The model is memorising specific phrasings and losing general capability. Symptoms include repetitive responses, narrow style range, and degraded performance on tasks similar to but not identical to the training distribution. The other options are off the standard regime entirely. Fractional epochs (A) leave the format adjustment incomplete. Dozens of epochs (C) or fifty plus (D) are pretraining-scale numbers; treating SFT as pretraining ignores that the base has already done the heavy lifting.

Quick reference

TL;DR

1 to 3 epochs is the standard SFT range; the base already knows language, so a few passes adjust format and style without overfitting the small curated set.

Memory aid

Easy to grasp

Think of an expert chef being taught to plate a specific new dish. They already know how to cook, hold a knife, and balance flavors from decades of experience. The new lesson is just about plating style: the specific arrangement, the garnish, the sauce drizzle. You show them the technique once or twice, they pick it up. If you make them practice the exact same plating fifty times in a row, they start producing identical-looking plates every time, even for dishes that should look different. They have stopped learning the principle and started copying the example. SFT works the same way. The model already knows English. One to three passes over the curated examples teach the format. Many more passes turn the model into a parrot.

Key concepts

Number of training epochs Supervised fine-tuning Overfitting Training hyperparameters

Detailed answer & concept explanation

~11 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

4 min: place the 1 to 3 epoch range, explain why the base model's pretraining sets the regime, describe the train versus held-out gap diagnostic, and dispatch the under- and over-training distractors.

Real-world examples

Real products, models, and research that use this idea.

Hugging Face TRL SFTTrainer default num_train_epochs is 3 across many recipe templates for Llama 4 and Mistral fine-tuning.
Llama 3 and Llama 4 instruction tuning at Meta uses one to two epochs on curated multi-million instruction sets, leaning toward single epoch on large mixes.
Tulu and Open Tulu recipes from Allen AI fine-tune on around 300K examples for two epochs as their published default.
DeepSeek V4 distillation recipes use two to three epochs on tens of thousands of student examples, with careful held-out monitoring to detect overfitting early.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does dataset size affect the optimal epoch count and where is the rough crossover?

QWhat diagnostics tell you that SFT has overfit, and which ones are unreliable?

QWhy does pretraining use far more epoch-equivalent passes than SFT, and when does that intuition break down?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating SFT as if it were pretraining and running it for dozens of epochs. The base model already knows language; SFT only adjusts format and style, and over-training memorises the demonstrations instead of learning the underlying patterns.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.