**save_strategy** in Hugging Face's `TrainingArguments` controls when a checkpoint is written during training. There are three legal values: `no`, `steps`, and `epoch`. The interesting choice is between `steps` and `epoch`.
With `save_strategy='steps'`, the trainer writes a checkpoint every `save_steps` optimizer updates. If you set `save_steps=500`, you get a checkpoint at step 500, 1000, 1500, and so on. The cadence is in units of training steps and is independent of how far you have moved through the dataset.
With `save_strategy='epoch'`, the trainer writes one checkpoint at the end of each full pass over the training set. If you train for three epochs you get three checkpoints, one per epoch boundary.
The two strategies save the **same artifacts**: model weights, optimizer state, learning-rate scheduler state, and RNG state. The only difference is timing.
The practical trade-off comes down to how long an epoch is. For short fine-tunes (a few thousand examples), `epoch` is convenient and produces few checkpoints. For long runs on large datasets (a single epoch might take hours), `steps` bounds how much progress a crash can lose to `save_steps * step_time`.
**save_strategy** is the Hugging Face `TrainingArguments` flag that controls **when** a checkpoint is written during training. The relevant choice is between `steps` and `epoch`.
**save_strategy='steps'** writes a checkpoint every `save_steps` optimizer updates. If `save_steps=500`, you get a checkpoint at step 500, 1000, 1500, and so on. The cadence is measured in optimizer steps, independent of where you are in the dataset.
**save_strategy='epoch'** writes one checkpoint at the end of each full pass through the training set. Three training epochs produce three checkpoints, one per epoch boundary.
The two strategies are **identical in what they save**: model weights, optimizer state (first and second moments for Adam), learning-rate scheduler state, and the random number generator state. The choice does not change which artifacts hit disk; it only changes the cadence.
**The practical trade-off is loss on crash.** A single epoch on a 50M-token dataset at 1024 sequence length and 16 effective batch size is roughly 3000 steps. At 5 seconds per step that is roughly 4 hours per epoch. With `epoch` cadence, a crash at minute 250 has you restarting from scratch. With `steps` cadence at `save_steps=500`, you lose at most 500 steps (about 40 minutes).
**Rule of thumb.** Use `steps` for any run where one epoch takes more than an hour; bound `save_steps` so the worst-case loss is acceptable for your operational tolerance (typically 30 to 60 minutes of recompute). Use `epoch` for short fine-tunes where one epoch is fast and the natural milestone is convenient. Three other flags are independent: `save_total_limit` caps disk usage by pruning old checkpoints, `save_only_model` strips optimizer state from on-disk saves, and `load_best_model_at_end` plus `metric_for_best_model` add best by metric selection on top.
Quick reference
TL;DR
save_strategy controls cadence only: steps saves every N optimizer updates, epoch saves at the end of each pass. Both save the same artifacts.
Imagine writing a long letter and choosing when to hit save in your word processor. One option saves every fifteen minutes regardless of where you are; the other saves only when you reach the end of a chapter. Both produce identical save files. The only difference is the timing. If the power goes out, the every fifteen minutes option loses at most fifteen minutes of work. The end of chapter option might lose hours if you were deep into a long chapter. That is exactly the trade-off in fine-tuning checkpoints. Step-based cadence bounds the worst-case loss; chapter-based cadence ties saves to natural data milestones. Pick steps for long runs, chapter-end for short ones.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Your fine-tune is short enough that one epoch finishes in minutes Step-based saves produce unnecessary disk churn; epoch cadence is cleaner and produces a tidy per-epoch artifact set.
You only care about the final model and never plan to resume mid-run Combine save_strategy with save_only_model=True and small save_total_limit, or set save_strategy='no' for purely exploratory runs.
You are running on truly preemption-free, fixed-duration hardware and a single epoch is short Crash protection adds little value when crashes are unlikely; epoch cadence keeps the artifact set small.
Quick scan · 2–4 min
Everything important, quickly.
5 min: what save_strategy controls + steps vs epoch cadence + identical artifacts saved + worst-case loss math + interactions with save_total_limit and save_only_model + independence from best by metric.
Side-by-side
Aspect
save_strategy='steps'
save_strategy='epoch'
Cadence trigger
Every save_steps optimizer updates
End of each full data pass
Artifacts saved
Model + optimizer + scheduler + RNG
Same
Best for
Long runs, large datasets, spot instances
Short fine-tunes, small datasets
Worst-case loss on crash
save_steps * step_time
Up to a full epoch
Disk pressure
Higher (more frequent saves)
Lower (one per epoch)
Real-world examples
Real products, models, and research that use this idea.
1TRL's SFTTrainer accepts the same save_strategy argument, with steps cadence the default in most community fine-tuning recipes.
2Axolotl exposes save_strategy in its YAML config, paired with save_steps and save_total_limit for typical multi-day runs.
3Hugging Face's autotrain default is save_strategy='steps' with save_steps tied to total step count for predictable artifact counts.
4DeepSpeed-integrated runs respect save_strategy and handle the cross-rank consolidation internally, transparent to the choice.
5Llama-Factory, LLaMA-Adapter, and other community tooling default to save_strategy='steps' for fine-tunes on large mixed datasets.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does save_strategy interact with eval_strategy in practice?
Sign in or go Pro to see the approach for this follow-up.
QWhy might you set save_only_model=True, and what does it cost?
Sign in or go Pro to see the approach for this follow-up.
QWhat is the right save_total_limit for a long pretraining-style fine-tune?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Reading the two strategies as different in what they save. They save the same thing (weights, optimizer, scheduler, RNG); only the cadence differs.