Walk through what actually happens during the prefill phase of an LLM forward pass.

When you send a prompt to an LLM, the very first thing the model does is process the entire prompt in one big pass. This phase is called **prefill**. Concretely, every token in the prompt goes through the embedding layer, then through every transformer layer of the model, in parallel. Inside each layer, the model computes attention across all the prompt tokens at once: each token attends to every previous token to gather context. The keys (K) and values (V) computed for every position are written into a data structure called the **KV cache**, where they are stored for the entire rest of the request. At the end of prefill, the model has done two things. First, it has built up the KV cache so that future steps do not have to recompute K and V for the prompt tokens. Second, it has emitted the first output token (the model is autoregressive, so the next token always comes out of the position immediately after the last input). After prefill finishes, the model enters the **decode** phase: a sequential loop that produces output tokens one at a time, each consulting the KV cache to attend to everything that came before. Prefill is one big parallel matrix multiplication over the entire prompt. Decode is many small sequential matrix-vector products. They have completely different performance characteristics, and good inference engineers think about them separately.

**Prefill** is the first phase of LLM inference: a single forward pass that processes the entire input prompt at once. Every prompt token is embedded, run through all transformer layers in parallel, and contributes its K and V projections to the KV cache. Attention is computed across all prompt positions simultaneously, and the model produces logits for the first output token at the end of the pass. The defining property of prefill is its **parallelism over the prompt length**. If the prompt is 2000 tokens, every layer's matmul has shape roughly `[2000, d_model] x [d_model, d_model]`. The 2000-token batch dimension means the weight matrix is reused 2000 times per multiply, so arithmetic intensity sits in the hundreds or thousands of FLOPs per byte. This pushes prefill onto the compute-bound side of the roofline: the GPU's tensor cores stay saturated and the workload runs near peak FLOPs. The other defining property is that **prefill populates the KV cache**. K and V tensors for every prompt position are computed once and stored in HBM. From that point on, the decode loop only computes K and V for the single new token at each step and appends to the cache. Without this caching, decoding the second token would require re-running prefill over the full prompt plus the first output token, which would multiply total compute by a factor of `T_output`. Prefill is the upfront investment that makes decode tractable. In production, prefill latency dominates time to first token, and TTFT is what the user perceives as 'how fast did the model start responding'. Optimizations like FlashAttention, prompt caching, and chunked prefill all target this phase.