What is the context window of an LLM?

The **context window** of a language model is the maximum number of tokens it can look at in one call. A token is a small chunk of text, roughly three quarters of an English word on average. This cap counts the prompt the user sends and the response the model produces, together. If a model has a 32k context and the prompt uses 30k tokens, the model has only 2k tokens left to reply with. The limit is fixed when the model is trained. The model learns where words sit using a positional encoding, and that scheme is only taught about positions up to the training maximum. Past that point the model has never seen a position before and tends to produce garbage. If you need more room than the model offers, the usual options are to truncate older content, summarise it before sending, retrieve only the relevant parts with RAG, or use a context-extension method like position-interpolation that stretches the encoding to longer ranges.

**The context window** is the maximum token count, prompt plus completion combined, that an LLM can attend to in a single forward pass. It is a hard ceiling baked in at training time, not a soft preference you can override at inference. The reason it is hard is that transformers learn token positions through a positional encoding scheme. The encoding is only trained on positions zero through the training maximum. Send the model token number 50,000 when it only ever saw up to 32,000 and the model has no idea where that token is. Quality collapses well before you hit the cap. In 2026 the typical floor is 128k tokens, with Gemini 3.1 Pro and Claude Opus 4.7 pushing into the million-token range. That capacity comes at a cost: KV cache memory grows linearly with sequence length, so long contexts dominate GPU memory and slow decode. When the input exceeds the window, the choices are truncation, summarisation, retrieval-augmented generation that pulls in only the relevant slices, or context-extension techniques like position-interpolation and YaRN that fine-tune the positional encoding to longer ranges without retraining from scratch.