Supervised Fine-Tuning (SFT)
Also known as: SFT, Instruction tuning
Fine-tune the base model on (prompt, ideal response) pairs; the first post-training step before alignment.
The first stage of post-training where a pre-trained base model is fine-tuned on labeled instruction-response pairs. Teaches the model what good answers look like before any preference-based alignment (RLHF or DPO).
In practice
Together with RLHF/DPO, SFT turns a raw next-token predictor into a chat assistant. Most alignment questions assume SFT happens first.
How it compares
SFT teaches the model what to say from labels; RLHF teaches it which of several outputs is preferred.
Related topics
Questions that mention this term
Related terms
DPO (Direct Preference Optimization)
Skip the reward model and PPO: fine-tune directly on preferred-vs-rejected response pairs.
Fine-tuning
Continue training a pre-trained model on task-specific data to specialize its behavior.
FSDP (Fully Sharded Data Parallel)
Shard weights+grads+optimizer states across GPUs and gather just-in-time; train huge models with limited per-GPU memory.
GRPO (Group Relative Policy Optimization)
PPO without the critic: advantages are computed by ranking multiple sampled responses against each other.
Instruction Tuning
Train a base model on many (instruction, response) pairs to make it follow natural-language commands.
Knowledge Distillation
Train a small student model to match a big teacher's outputs: cheap, fast inference with most of the quality.