Glossary · Foundations

Context Window

Also known as: Context length, Sequence length

The max number of tokens a model can attend to at once.

The maximum number of tokens a model can process in a single forward pass. Determines how much text can be included in a prompt. Modern models range from 4K to 1M+ tokens.

In practice

Drives RAG vs long-context trade-offs and inference cost. Senior questions cover RoPE scaling, attention's quadratic cost, and lost-in-the-middle.

Comparisons that include Context Window

Hallucination Mitigation vs Context Window ManagementHallucination is a correctness problem and context window a capacity one; in RAG you tune retrieval to solve both at once, ground the answer without overflowing the prompt.Long context vs RAGLong context lets the model see everything; RAG selects what it should see. Long context is convenient but not free: cost scales, and quality falls off past the middle of the window.

Related terms

Attention Mechanism

How a model decides which input tokens to weight when computing each output token.

Retrieval-Augmented Generation (RAG)

Fetch relevant docs at inference time and stuff them into the prompt so the model can answer from real data.

Transformer

The attention-only neural architecture behind GPT, Claude, Gemini, and almost every modern LLM.

Chunking

Cutting documents into retrievable pieces before embedding them.

Hallucination

When a model confidently makes up something that isn't true.

Mixture of Experts (MoE)

Scale model capacity by routing each token to a small subset of expert MLPs instead of using all of them.