Explain why context engineering emerged as a distinct discipline in 2025-2026

Short answer·Easy·4.0 · 0·~3 min·Asked atAnthropic OpenAI Google

Attempt it

Why is context engineering treated as a separate practice from prompt engineering now, when a few years ago both fell under one bucket? Give two or three concrete drivers.

Free · 2 AI evals / day

In 2022 and early 2023, the phrase 'prompt engineering' covered the entire job of getting good behavior out of a language model. The work was almost entirely about how to phrase the instruction. By 2025-2026 the work has split into two clear disciplines, and three concrete drivers caused that split. **Retrieval and tool use became table stakes.** Almost every non-trivial application in 2026 pulls in retrieved chunks from a vector store, runs tool calls, or both. A typical RAG application sends 2K-10K tokens of retrieved content per turn. A typical agent application accumulates tool outputs over many turns. The instruction itself is a small fraction of what reaches the model, and the rest is composed at runtime from databases, search results, tools, memory, and history. Tuning the instruction in isolation cannot fix problems that live in that composition. **Context windows grew, but effective accuracy did not keep up.** In 2022 a context window was a few thousand tokens. In 2026 frontier models advertise 200K to 2M token windows. But the empirical evidence, the lost-in-the-middle finding (Liu et al. 2023), long-context benchmarks like RULER (2024) and BABILong (2024-2025), consistently shows accuracy degrading well before the hard limit. The result is that there are now two budgets, not one: the hard limit the API will accept, and the soft budget beyond which quality starts to fall. Managing the soft budget is a first-class engineering concern that did not exist when the hard limit was 4K. **Agentic loops produce long trajectories with state.** Agents that call multiple tools across many turns produce trajectories whose history, scratchpad, plan, and tool results all need to be selectively kept, summarized, or evicted. This is state management across iterations, not instruction writing. The skills that make a good agent loop work, placement, eviction, summarization, cache-aware design, are not prompt-engineering skills. The common thread across all three drivers is that the work shifted from the words inside the instruction to the system that composes the full input. That is what the new term names.

Three drivers, each with a concrete signature in how production LLM applications are built. **Retrieval and tool use became default architecture.** In 2023, RAG and tool use were emerging patterns; by 2026 they are table stakes for almost any non-trivial application. A modern call to a frontier model typically includes retrieved chunks from a vector store (2K-10K tokens, sometimes more), tool outputs from earlier steps, and persona facts from a memory store. The instruction itself is often a small fraction, sometimes under 5%, of what reaches the model. This shifts the locus of design from *how is the instruction phrased* to *what gets composed into the input on this turn*. The vector store, the rerank pipeline, the tool wrappers, the memory store, the schema definition, none of these are prompt-string problems, and none of them are solvable by tuning the instruction. **Context windows grew but effective accuracy did not.** From 4K in early 2023 to 200K-2M in 2026, the advertised context limit has grown by orders of magnitude. The empirical reality is more constrained. Lost-in-the-middle (Liu et al. 2023) showed that LLM attention is U-shaped over long contexts and content in the middle gets attention-starved. Subsequent long-context evals, RULER, BABILong, NoCha, replicated and extended this finding across every modern frontier model. The implication is that the *effective* context budget is much smaller than the *hard* limit, and the soft budget is task-dependent, model-dependent, and usually a fraction of the advertised number. Managing the soft budget, deciding what goes in, what gets evicted, what gets summarized, where each slot is placed, is a first-class engineering discipline that did not exist when budgets were small enough to use everything that fit. **Agentic loops produce long stateful trajectories.** Agent applications run for tens or hundreds of iterations, with each iteration adding tool calls, observations, scratchpad notes, and plan

Quick reference

TL;DR

Three drivers forced the split: retrieval and tool use made the instruction a small fraction of the input, context windows grew but effective accuracy did not keep up, and agent loops produced long trajectories whose

Memory aid

Easy to grasp

Picture how cooking changed over a few years. In 2022 a chef worked from one recipe card with everything they needed to know on it. By 2026 the same chef has a pantry of a thousand ingredients delivered fresh every morning, a brigade of sous-chefs handing things over, a notebook of what regular customers like, and a counter that fits only so many ingredients at once. The recipe card is still important, but writing a perfect card no longer matters as much as managing the pantry, the brigade, and the counter. That is exactly what happened to LLM applications: the instruction is still important, but the work shifted to managing everything around it.

Key concepts

Context Engineering Prompt Engineering

Detailed answer & concept explanation

~8 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

Name the three drivers and explain each with a concrete signature. Retrieval and tool use as default architecture: instruction becomes a small fraction of the input. Context windows growing faster than effective accuracy: lost-in-the-middle, RULER, BABILong establish that the effective budget is smaller than the nominal budget. Agent loops producing trajectory-level state: history, plans, observations, all needing eviction and summarization. Close with the framing: system design versus string wording, and the historical note that the vocabulary stabilized in 2025-2026 to acknowledge work that had been happening for years.

Real-world examples

Real products, models, and research that use this idea.

The rise of RAG-first products like Perplexity (2022 founding, mainstream 2024-2025) where the instruction is short and the heavy lifting is in retrieval and reranking.
Claude Code and Cursor (2024-2026) where the system prompt is hand-tuned but the engineering value lives in tool wrappers, file-context selection, and trajectory-state management.
LangGraph's typed-state pattern (2024-2025) which formalized trajectory-level state as a first-class concept distinct from prompt content.
Mem0, Letta, and Zep (2023-2026) as purpose-built memory frameworks that solve a problem prompt engineering does not even pose.
The shift from format by example prompting to Pydantic / Zod schema-driven structured output across the major SDKs (2024-2026).

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QDid model capability changes drive the split, or did application architecture drive it?

QWhat would a counterfactual look like where the split never happened?

QAre there sub-disciplines emerging within context engineering that might split off next?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Pointing at bigger context windows as the only driver, large windows did not solve the problem, they just made the budget question harder by hiding the effective-window limit behind a generous hard limit.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.