Concepts · Foundations

Context Window

Also known as: Context length, Sequence length

The max number of tokens a model can attend to at once.

The maximum number of tokens a model can process in a single forward pass. Determines how much text can be included in a prompt. Modern models range from 4K to 1M+ tokens.

In practice

Drives RAG vs long-context trade-offs and inference cost. Senior questions cover RoPE scaling, attention's quadratic cost, and lost-in-the-middle.

Comparisons that include Context Window

Hallucination Mitigation vs Context Window ManagementHallucination is a correctness problem and context window a capacity one; in RAG you tune retrieval to solve both at once, ground the answer without overflowing the prompt.Long context vs RAGLong context lets the model see everything; RAG selects what it should see. Long context is convenient but not free: cost scales, and quality falls off past the middle of the window.

Questions that mention this term

Related terms

AI System Design

End-to-end design of production LLM systems: ingestion, retrieval, serving, eval, monitoring.

Attention Mechanism

How a model decides which input tokens to weight when computing each output token.

BM25

Classical TF-IDF-style lexical scoring: a surprisingly strong baseline, still the first stage of many hybrid RAG pipelines.

Causal Mask

Attention mask that hides future tokens so each position can only see itself and prior tokens.

Chunking

Cutting documents into retrievable pieces before embedding them.

Context Engineering

Deciding what makes it into the model's context window (prompt + RAG + history + memory), and why.