An SFT run shows a striking pattern: loss drops smoothly within each epoch, then jumps back UP at the moment a new epoch starts, producing a sharp saw-tooth aligned exactly with epoch boundaries. Training is otherwise stable. Identify the most likely cause, why the jumps happen specifically at epoch boundaries, and what one-line fix usually resolves it. Also name the secondary suspect to rule out.
Free · 2 AI evals / day
**The pattern of a saw-tooth aligned exactly with epoch boundaries** is almost always a data-order problem, not a training-instability one.
The most likely cause is that the data loader is not shuffling between epochs, or the shuffle seed is fixed across epochs. Each epoch presents batches in the same order. Within an epoch the optimiser adapts to that specific sequence and loss falls smoothly. When the next epoch starts, the loader hands back batch one again, which is harder than the last batch of the previous epoch, so loss jumps up. The jumps happen at epoch boundaries because the boundary is the reset of the iteration order.
The model is still learning. Within-epoch loss is still falling, often slightly faster each epoch as the model gets better. The saw-tooth is a bookkeeping artifact of measuring loss on a non-stationary sample order.
The one-line fix is to enable per-epoch shuffling with an epoch-dependent seed. In PyTorch, set shuffle equal to True in the DataLoader. In distributed training with a DistributedSampler, you also need to call sampler.set_epoch(epoch) at the start of each epoch, otherwise the sampler reuses the same shuffle every epoch.
The secondary suspect to rule out is a curriculum or sort by length sampler that re-sorts at the top of every epoch. That gives the same visual pattern. Cosine restarts in the learning-rate scheduler usually produce drops at the boundary, not jumps up.
**The diagnosis** is a data-order artifact, almost always missing per-epoch shuffling or a fixed shuffle seed. Each epoch presents batches in exactly the same order, so the optimiser smooths the loss through the sequence within an epoch, and when the next epoch starts the loader hands back batch one again. Batch one is harder than the final batch of the previous epoch (the model was tuned to those last examples just before the boundary), so loss jumps up. The saw-tooth aligns precisely with epoch boundaries because the boundary is the reset of the iteration order.
**Why it is a measurement issue, not a training issue.** Within each epoch the curve still falls. Successive epochs typically reach lower per-batch loss values than previous epochs at the same iteration position, which is the signal that the model is still learning. The saw-tooth is the optimiser passing through the same batch sequence again on a slightly better model, not a sign of instability.
**The fix.** Enable per-epoch shuffling with an epoch-dependent seed. In PyTorch's DataLoader, shuffle equal to True with the default sampler does the right thing for single-GPU training. In distributed training the DataLoader uses DistributedSampler, and you must call sampler.set_epoch(epoch) at the top of each epoch loop, otherwise the sampler reuses the same shuffle every epoch and the saw-tooth persists even with shuffle equal to True. This is the one-line fix that teams forget under FSDP or DDP.
**Secondary suspect to rule out.** A curriculum or sort by length sampler that re-sorts at the top of every epoch produces the same visual pattern. Check the sampler config. Cosine-restart learning-rate schedulers usually produce drops at the boundary, not jumps up, so the direction of the discontinuity tells you which suspect to chase first.
Quick reference
TL;DR
The data loader repeats the same batch order each epoch, so loss restarts at the same hard-batch sequence. Fix: per-epoch shuffling with an epoch-dependent seed, plus sampler.set_epoch under DDP or FSDP.
Picture practising a piano programme of twenty pieces in the same fixed order every day. Within a day you warm up on the first piece, get smoother by the tenth, and play the last few beautifully. Tomorrow you start again on piece one, which is hard from cold, and your score for that piece looks worse than where you ended yesterday. The score chart looks like a saw, jumping up every morning. The fix is to shuffle the order each day so you do not always restart on the same cold piece. In a training run, the same trick is to reshuffle the data on every full pass so the first chunks are not always the same hard ones.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Single-batch or pure online streaming training where epochs are not a concept The saw-tooth diagnosis is keyed to epoch boundaries, with no epochs there is no boundary to anchor the pattern, the diagnostic does not apply.
Deliberate curriculum training where ordering is part of the recipe Curriculum runs intentionally re-sort each epoch from easy to hard, the saw-tooth is expected, not a bug, treating it as a loader problem would break the curriculum.
Single-epoch training runs where no second boundary exists With one epoch there is no second epoch boundary to produce a repeated saw-tooth, any spike has a different cause that needs different diagnosis.
Quick scan · 2–4 min
Everything important, quickly.
4 min: identifying the saw-tooth as a data-order artifact + why same batch order produces same curve + the one-line shuffle fix + the DistributedSampler.set_epoch gotcha + curriculum and cosine-restart suspects to rule out + why it is a measurement issue, not a training issue.
Real-world examples
Real products, models, and research that use this idea.
1PyTorch's DistributedSampler documentation explicitly warns that set_epoch must be called or shuffling will repeat, a warning many teams discover only after seeing this curve.
2Hugging Face Trainer handles set_epoch automatically since transformers 4.x, but custom training loops with raw PyTorch DDP often miss it.
3Axolotl and LLaMA-Factory both wire set_epoch correctly by default in 2026, which is why community fine-tunes of Llama 4 and Qwen rarely show this saw-tooth despite running at multi-GPU scale.
4DeepSpeed's documentation calls out the same gotcha, and several debugging guides for 2026 reasoning-model training open with this pattern as the first diagnostic to check.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you distinguish the data-order saw-tooth from a cosine-restart artifact if both happen to align with epoch boundaries?
Sign in or go Pro to see the approach for this follow-up.
QWhy does set_epoch matter for DistributedSampler when shuffle equal to True is already set in the DataLoader?
Sign in or go Pro to see the approach for this follow-up.
QCould this saw-tooth pattern ever indicate a real problem rather than a loader artifact?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Chasing this as a training-instability bug. It is a data-order artifact, the model is still learning, the loss curve is just being measured on a non-stationary sample order. Look at the loader, not the optimiser.