Describe what 'autoregressive' means for LLM generation.

**Autoregressive** is a word from statistics: a process where each new value is predicted from past values of the same series. For LLMs, the series is the sequence of tokens being generated, and each new token is sampled conditioned on every token before it, both the prompt tokens and the tokens the model already emitted. The per-step loop has four parts. **Forward pass.** The model runs one forward pass through all its layers, taking the current context (prompt plus all generated tokens so far) and producing a probability distribution over the entire vocabulary. **Sample.** The decoder picks one token from that distribution; greedy decoding picks the highest-probability token, temperature sampling and top-p sampling pick stochastically from a softened distribution. **Append.** The sampled token is appended to the context. **Check.** The loop checks termination conditions and either repeats or stops. The loop terminates when one of three things happens. The model samples the end-of-sequence token (often called EOS or end-of-text), which it learned to emit at natural stopping points. The output contains a stop sequence the caller specified (like a specific newline pattern). Or the total tokens generated reaches the `max_tokens` cap. The practical consequence is that generation is inherently sequential: you cannot compute token T+1 until you have token T. This sequential dependency is the fundamental reason LLM decode is slow and bandwidth-bound, and it is the reason for every optimization that tries to break the chain: speculative decoding, parallel decoding heads, draft models, multi-token prediction.

**Autoregressive generation** means each new token is sampled conditioned on every previously generated token. The model factors the joint probability of a sequence as the product of conditionals: `P(t1, t2, ..., tN) = prod P(ti | t1...ti-1)`. Generation realizes that factorization left-to-right one token at a time. The **per-step loop** has four stages: 1. **Forward pass.** The model takes the current context (prompt plus all generated tokens so far) through all transformer layers, producing logits over the full vocabulary at the final position. A softmax converts the logits to a probability distribution. 2. **Sample.** The decoder picks one token. Greedy picks argmax, sampling picks stochastically based on temperature, top-k, top-p, or other modifiers. 3. **Append.** The sampled token is concatenated to the context and the KV cache absorbs its K and V tensors so the next step does not recompute earlier projections. 4. **Terminate or repeat.** Check stop conditions; if none fire, run another step. **Termination conditions** are three: the model samples its trained end-of-sequence token (EOS), the generated text matches a caller-supplied stop sequence, or the total tokens produced reaches `max_tokens`. Production servers track all three. The **performance consequence** of autoregression is the dominant fact of LLM serving. Token T+1 depends on token T, so generation cannot parallelize across the output dimension. Every decode step is a full forward pass through all layers for a single new token, which is why decode is matrix-vector at batch 1, why it sits at arithmetic intensity near 1, and why it is HBM-bandwidth-bound. Every modern serving optimization, from continuous batching through speculative decoding to multi-token prediction, attacks the consequences of autoregression in some way without breaking the conditional dependency the model was trained on.