Sequence length vs context window, what is the practical distinction and which one drives KV cache size?

**Context window** is the architectural ceiling. It is fixed when the model is trained and shipped. Claude Opus 4.7 supports 1M tokens; GPT-5.5 supports 400k; Llama 4 Maverick supports 1M. The number cannot grow without retraining or extending positional encoding. **Sequence length** is the live count: prompt tokens plus output tokens generated so far for one specific request. A user asking a 200-token question and getting a 300-token reply has a sequence length of 500. The KV cache is the per-request memory that stores the key and value vectors for every past token, so attention does not have to recompute them. It scales linearly with sequence length, not with context window. This is the practical takeaway. A serving stack on a 1M-context model still only pays for the tokens the request actually holds. Most chat turns are 500-2000 tokens, so most KV cache footprints are tiny relative to the model's advertised ceiling. You only pay 1M tokens of cache when someone actually sends a million-token prompt.

**Sequence length** is per-request and grows token by token. **Context window** is per-model and fixed at training time. The KV cache cost on the GPU follows the first number, not the second. The formula for KV cache memory makes it concrete: $$\text{bytes} = 2 \cdot L \cdot H_{kv} \cdot d_h \cdot T \cdot b$$ Here `L` is layers, `H_kv` is KV heads, `d_h` is head dim, `T` is the live sequence length, and `b` is bytes per element (1 for FP8, 2 for BF16). The cache grows linearly in T. The context window only appears as the maximum that T is allowed to reach. In production this distinction matters because serving costs scale with realized T, not with advertised ceiling. A model that supports 1M tokens does not pre-allocate 1M tokens of cache. Paged-attention engines like vLLM allocate KV pages on demand as sequences grow, and free them when the request ends. Where it bites: long-context features like full-document analysis or long agent traces push T much higher. A single 200k-token request on a 70B model with grouped-query attention burns roughly 4-8 GB of KV cache by itself in FP8, which crowds out batch concurrency. The lever is T, and the cost lens should be T.