**Answer B is correct.** Teacher forcing is the standard way transformer language models are trained on supervised data. At every position in the sequence, the model is conditioned on the gold previous tokens (the tokens from the training example), not on tokens it generated itself.
This design choice has a huge practical payoff. Because each position is conditioned on gold context, the predictions at different positions are independent given the input. That means the model can score the whole sequence in a single parallel forward pass through the transformer. Every position predicts the next gold token simultaneously, and cross-entropy is summed across all positions.
If you did not use teacher forcing, you would have to generate token by token, plugging each generated token into the next step's input. That would be hundreds of times slower per training example and would also chase a moving target, since the input distribution at each step depends on the model's own (initially terrible) outputs.
The three wrong options describe different things entirely. Option A is scheduled sampling, option C is knowledge distillation, option D is fiction.
**Option B.** Teacher forcing means that at every position in the training sequence, the input fed to the model is the gold previous token from the training example, not a token the model generated. This is the standard training recipe for autoregressive transformers.
The property that matters is parallelism. With gold context at every position, the model can produce next-token predictions for the entire sequence in a single forward pass. The transformer's causal attention mask ensures each position only sees prior tokens, so the prediction at position t depends on positions 0 through t-1, all of which are gold. Cross-entropy is computed at every position and summed, giving the full loss in one shot.
Without teacher forcing, you would need to generate token by token, feeding each generated token back as input for the next step. That serial process is hundreds of times slower and also chases a moving target during early training, when the model's outputs are noisy.
The distractors describe related but distinct concepts. Scheduled sampling (option A) mixes gold and sampled context to bridge train-test mismatch. Knowledge distillation (option C) is about what targets the student matches, not what previous tokens condition each step. Option D invents a mechanism that does not exist.
Quick reference
TL;DR
Teacher forcing means each training position is conditioned on the gold previous tokens, not on the model's own predictions, so the whole sequence is scored in one parallel forward pass.
Imagine teaching a kid to copy a sentence one word at a time. Two ways to do it. The first way, you let the kid write a word, then write the next word based on whatever they wrote, even if their first word was wrong. The second way, you erase any mistake immediately and let them continue from the correct word every time. Teacher forcing is the second way. The kid always sees the right setup, so they learn what comes next from a correct starting point at every step. It also lets you grade every word of the whole sentence at once, instead of waiting for one to be done before moving to the next.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
You are training a small RNN that explicitly wants to learn recovery from its own errors Scheduled sampling or free-running training can be useful here, though it adds complexity that rarely pays off for large transformers.
You are doing reinforcement learning from human feedback or DPO These methods generate rollouts from the model itself and score them against a reward signal; teacher forcing does not apply at the policy-update step.
You are training a non-autoregressive model like a diffusion language model The whole sequence is denoised in parallel from noise, so there is no notion of previous-token conditioning to force.
Quick scan · 2–4 min
Everything important, quickly.
3 min: state the gold-context rule, explain the parallel forward pass payoff, dismiss the three distractors as different concepts entirely, and end on the exposure-bias caveat.
Real-world examples
Real products, models, and research that use this idea.
1Every Hugging Face Trainer SFT run on Llama 4 Maverick in 2026 uses teacher forcing by default; the training loop never calls model.generate.
2DeepSpeed and FSDP both rely on teacher-forced parallel forward passes; their throughput numbers assume this mode.
3Claude Opus 4.7 and Gemini 3.1 Pro SFT recipes use the same conditioning rule; teacher forcing is universal across modern transformer training.
4Bengio et al.'s 2015 Scheduled Sampling paper proposed an alternative for RNNs, but the approach did not transfer well to large transformers and is rarely used in 2026 LLM workflows.
5Unsloth and Axolotl training notebooks both rely on teacher-forced batches, and their packing optimisations only work because the entire sequence is scored at once.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWalk through what exposure bias means and how RLHF or DPO compensates for it.
Sign in or go Pro to see the approach for this follow-up.
QHow does the causal attention mask cooperate with teacher forcing to produce the parallel forward pass?
Sign in or go Pro to see the approach for this follow-up.
QWhy did scheduled sampling never become the default for large transformer SFT?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Confusing teacher forcing with knowledge distillation. They are orthogonal: teacher forcing is about which previous tokens condition each step, distillation is about what targets the student matches.