Adam

Paged optimizer in QLoRA, what gets paged, and to where?

Why ramp the learning rate during warmup instead of starting at the peak value?

Gradient norm clipping, name the failure mode it exists to prevent

Adam optimizer state per parameter: fill in the moment buffers and bytes

Estimate peak VRAM for a 7B full FT run in bf16 + Adam

Fill in the full FT memory contributions for a 7B model