Match the model class to the context-engineering knob it most rewards

Match pairs·Medium·4.0 · 0·~2 min·Asked atAnthropic OpenAI Google

Attempt it

Small fast model with 8k window

Frontier model with 200k window and prompt caching

Reasoning model that emits long internal traces

Open-weight model with weak long-context performance

Vision-capable model with images in context

Context engineering is not one playbook. It is a small set of rules and you pick which one matters most based on the model you are calling. **Small fast model with an 8k window.** The dominant constraint is space. You cannot afford to be generous with chunks. Tight top-k (3-5), aggressive trimming of low-value content, and a rerank step are non-negotiable. Without rerank, the precious 8k tokens fill with mediocre context. **Frontier model with a 200k window and prompt caching.** Space is no longer the constraint; cache stability is. The system prefix should be byte-stable across turns so cache hits compound. Dynamic content lives past the cache boundary. **Reasoning model that emits long internal traces.** Models like o-series and Claude with extended thinking produce long internal chains before the user-visible output. You must reserve a large output budget. Filling the input does not leave room for the model to reason. **Open-weight model with weak long-context performance.** Even though the window is nominally long, the U-curve is steeper. Place the most important chunks at the head and the tail; treat the interior as cheap seats. **Vision-capable model with images in context.** Images consume real tokens (often hundreds to thousands per image). Account for them separately and place them adjacent to the text they describe.

Model class drives layout. Five common classes, one dominant knob each. **Small fast model, 8k window.** Every token matters. The mandatory moves are aggressive trimming, tight top-k (3-5 chunks max), a cross-encoder rerank to make those few chunks count, and trimming low-value boilerplate from the system prompt. Skipping rerank here is the single biggest mistake; the model has no slack to absorb mediocre context. **Frontier model with prompt caching, 200k window.** Cache hit rate is the highest-leverage knob in the stack in 2026. Anthropic Claude Opus 4.7, OpenAI GPT-5.5, and Gemini 3.1 Pro all reward a byte-stable system prefix with deep discounts (often 90% off input cost) and faster TTFT. The layout discipline is: static persona, policies, and tool definitions in a frozen prefix; everything time-sensitive past the cache boundary. **Reasoning model.** Extended-thinking models produce long internal traces. The input you provide competes for output token budget. The fix is to reserve a generous output reservation (often 4-8k or more) and resist stuffing the input. Quality on reasoning models often improves when input shrinks. **Open-weight model with weaker long-context.** Llama 4 Maverick and similar open-weight 2026 models nominally support 128k-plus contexts but the lost-in-the-middle curve is steeper than frontier closed models. Head and tail layout: rank-1 chunk first, rank-2 chunk last, weaker chunks in the interior. Treat the middle as cheap seats. **Vision-capable model.** Images are not free. Each image consumes hundreds to thousands of tokens depending on resolution. Budget for them explicitly. Place each image immediately adjacent to the text that references it; remote placement weakens the model's ability to ground.

Quick reference

TL;DR

Different model classes reward different context knobs: small windows want aggressive trimming, cached prefixes want stability, reasoning models want output reservation, weak long-context models want head and tail

Memory aid

Easy to grasp

Imagine packing for five different trips. A weekend carry-on (small window) means ruthless trimming, only essentials. A road trip with a passenger who has a magic suitcase that remembers your packing list (cached prefix) means stable layout to keep the magic working. A hike where you will write a long journal at the top (reasoning model) means leaving lots of paper blank for later. A drive where you tend to only remember the first and last hour (weak long-context) means putting the important stops at the start and end. A photo trip (vision model) means keeping camera notes right next to the photos. The packing rules are not universal. They depend on the trip.

Key concepts

Context engineering Inference optimization Retrieval-augmented generation

Detailed answer & concept explanation

~4 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

5 minutes: name the five classes and the dominant knob for each; explain why prompt caching rewards stability and reasoning rewards restraint; describe head and tail layout.

Real-world examples

Real products, models, and research that use this idea.

Anthropic's published Claude Opus 4.7 prompt-caching guide shows up to 90% input-cost reduction and 85% TTFT reduction for stable cached prefixes; the load-bearing optimization for that model class.
OpenAI's GPT-5.5 reasoning configuration recommends reserving output token budgets of 4k+ for non-trivial tasks; documented in the responses API guide.
Llama 4 Maverick 2026 long-context benchmarks (RULER, BABILong) confirm a steeper midcontext dip than frontier closed models, validating head and tail layout for that fleet.
Gemini 3.1 Pro vision token accounting: images are charged at provider-defined token rates that must be budgeted alongside text.
Cursor and Continue route different file-edit tasks to different models in 2026 with per-class context budgets, illustrating the mixed-fleet adapter pattern.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow do you design a router that picks the model class first, then the context layout?

QWhat goes wrong if you mix a cached prefix with retrieval results above the cache boundary?

QHow does the head and tail layout interact with prompt caching?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Applying the same context layout across every model in your fleet. A layout tuned for Claude Opus 4.7 with caching is a poor layout for a small 8k local model and vice versa.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.