Instruction Tuning
Also known as: Instruction fine-tuning
Train a base model on many (instruction, response) pairs to make it follow natural-language commands.
Fine-tuning a pre-trained LLM on diverse (instruction, response) pairs so it follows natural-language commands. Closely overlaps with SFT; "instruction tuning" emphasizes broad task diversity over task-specific specialization.
In practice
Step that turns GPT-3 into ChatGPT, conceptually. Interviews probe data quality, task mixture, and the link to RLHF.
How it compares
Instruction tuning is broad-coverage SFT; SFT is the general supervised post-training step.
Related topics
Questions that mention this term
- Does modern LLM fine-tuning default to causal LM or masked LM, and why?
- What is the sensible num_train_epochs for a typical instruction SFT run on a curated dataset?
- You are fine-tuning on Python code only. Should you reuse the base tokenizer, train a new one, or extend the vocab?
- Why does completion only…
- How does a base model differ from an instruction tuned model?
- What is supervised fine-tuning (SFT)?
Related terms
DPO (Direct Preference Optimization)
Skip the reward model and PPO: fine-tune directly on preferred-vs-rejected response pairs.
Fine-tuning
Continue training a pre-trained model on task-specific data to specialize its behavior.
FSDP (Fully Sharded Data Parallel)
Shard weights+grads+optimizer states across GPUs and gather just-in-time; train huge models with limited per-GPU memory.
Knowledge Distillation
Train a small student model to match a big teacher's outputs: cheap, fast inference with most of the quality.
LoRA (Low-Rank Adaptation)
Fine-tune by training small low-rank update matrices instead of every weight: same quality, ~1% of the cost.
Pre-training
Self-supervised next-token training on a huge unlabeled corpus, producing the base model.