Define LLM inference and explain how it differs from training.

**LLM inference** is what happens every time you send a prompt to a model and get a reply back. The model does a forward pass: it reads the input tokens, multiplies them through its layers using the weights it already has, and produces output tokens one at a time. Nothing about the model changes during this process. Training is what created those weights in the first place. During training, the model processes labeled examples, compares its predictions to the correct answers, computes how wrong it was (the loss), then backpropagates gradients to nudge every weight slightly toward the correct answer. Training needs gradients, an optimizer, and often three or four times more GPU memory than inference because it has to hold activations from the forward pass to use during the backward pass. The cost picture is asymmetric. Training a frontier model can cost tens of millions of dollars but happens once. Inference is cheap per request but happens billions of times per day across all users. That is why inference optimization (faster decoding, smaller KV cache, better batching) is where most production engineering effort goes.

**Inference** is the forward-only execution of a trained model: weights are frozen, no gradients are computed, no parameters get updated. The input prompt goes through every layer, attention is computed against the cached keys and values of prior tokens, and output tokens are sampled one by one until a stop condition fires. Training is structurally different. It requires labeled data, computes a loss against the targets, runs the backward pass to compute gradients, holds the optimizer state (typically Adam's first and second moments), and updates the weights. The memory footprint is roughly 4x the parameter count once you add activations, gradients, and optimizer state, versus roughly 2x for inference (weights plus KV cache). The economic split is what makes this an interview question. Training a frontier model is a one-time bill measured in millions of GPU-hours. Inference is the recurring bill, growing linearly with traffic, and it dominates the lifetime cost of any deployed model. A 2-billion-parameter open-weight model might cost $100k to train but burn $5M per year on inference if it serves a large product. That asymmetry is why inference-side techniques like quantization, paged attention, continuous batching, and speculative decoding get so much engineering attention.