Context Engineering
Also known as: Context management
Deciding what makes it into the model's context window (prompt + RAG + history + memory), and why.
The discipline of optimizing what information ends up in an LLM's context window (including prompt structure, retrieval, summarization, history management, and memory) to maximize task quality within token and latency budgets.
In practice
The umbrella discipline above prompt engineering and RAG. Senior LLM app design interviews increasingly frame the problem this way.
How it compares
Prompt engineering tunes the static prompt; context engineering manages everything that flows in dynamically.
Related topics
Questions that mention this term
- In a production LLM app with a stable system prompt + long retrieved context + variable user query, which part of the prompt becomes cacheable via Anthropic/OpenAI prompt caching?
- What is few shot prompting and what does the 'few' refer to?
- Where should an agent scratchpad live in the context, and what gets pruned from it each iteration?
- List the trace spans a context engineering pipeline should emit per call
- Compare episodic and semantic long term memory with examples from a personal assistant agent
- Which failure pattern points to chunk size being too large rather than too small?
Related terms
AI System Design
End-to-end design of production LLM systems: ingestion, retrieval, serving, eval, monitoring.
BM25
Classical TF-IDF-style lexical scoring: a surprisingly strong baseline, still the first stage of many hybrid RAG pipelines.
Chunking
Cutting documents into retrievable pieces before embedding them.
Context Rot
Long-running chats degrade: early instructions get forgotten, and tool calls become less reliable.
Context Window
The max number of tokens a model can attend to at once.
Fixed-size Chunking
Slice documents every N tokens with a small overlap. Fast and format-agnostic, but cuts through headings and mid-topic.