Which of these is actually saved when a fine-tuning checkpoint is written?

A **training checkpoint** is the saved state that lets you resume an interrupted run without losing meaningful progress. It contains far more than just the model weights. The weights are the obvious part: the current values of every trainable parameter. But to actually continue training cleanly you need three more pieces. First, the optimizer state. Adam keeps two running averages per parameter, called m and v, that smooth gradient updates over time. If you reset these to zero on resume, Adam starts fresh and produces a visible loss spike for the next several hundred steps as it rebuilds the averages. Second, the learning-rate scheduler state. The scheduler tracks where you are in a warmup or decay curve. Lose this and the LR jumps back to the schedule's starting value. Third, the random number generator state and the step counter. These let the dataloader reproduce the exact same shuffle order and skip ahead to where you stopped, so you do not retrain on examples you already saw. Save all four together and resume is essentially seamless. Skip any one of them and the loss curve shows a visible bump at the resume point.

**A checkpoint** persists the full mutable state of a training run, not just the model. The standard contents are: model weights, optimizer state, learning-rate scheduler state, RNG state, and the current step or epoch counter. The optimizer state is the biggest single contributor by size. Adam stores two fp32 buffers per trainable parameter, the first and second moment estimates, which together are about 8 bytes per parameter. For a 7B model that is 56 GB of optimizer state on top of 14 GB of bf16 weights, which is why checkpoints for full fine-tuning are large. The scheduler state is small but critical. Warmup then decay schedules track the current step inside the schedule, and resuming from step zero would replay the warmup period and disturb convergence. The RNG state plus the step counter lets the dataloader reconstruct exactly where it was in the shuffled stream. Most modern trainers do not embed the data itself in the checkpoint; instead they re-create the dataloader with the saved seed and fast-forward by the step count. Resume quality is the test: a properly saved checkpoint resumes with no visible discontinuity in the loss curve. A weights-only checkpoint resumes with a sharp spike as Adam rebuilds its moments. That spike is the diagnostic that someone saved the wrong thing.