Pick the right fallback when the assembled prompt overflows the model's context window

When the assembled prompt is bigger than the model's context window, something has to go. The question is which of four common approaches is the right one. **Option 0, reject the request.** Sometimes correct (if the user pasted a 100k-token document), but usually wrong. Most overflow is caused by retrieval volume, summary growth, or accumulated tool results, things the user did not control. Punishing the user for the system's assembly is user-hostile. **Option 1, silently truncate from the start.** Catastrophic. The start of the prompt is the system prompt, persona, safety rules, and tool definitions. Head truncation decapitates the agent. Symptoms: agents start ignoring brand voice or making unsafe responses under load. Never the right move. **Option 2, apply a documented eviction order.** Correct. The order is declared in advance: sacred tiers (system prompt, current user question, output reservation) are protected; evictable tiers (extra retrieved chunks, lower-level summaries, the rolling summary, then recent verbatim turns as a last resort) are trimmed in declared order. The system fails gracefully and the postmortem is debuggable. **Option 3, silently swap to a larger-window model.** Tempting but wrong. The caller did not authorize a model swap. Price changes (a 1M-window model may be 3-5x more expensive). Behavior changes (different model snapshots produce different outputs). The audit trail breaks. Model selection must be an explicit policy, not an emergency fallback. The correct answer is option 2 because it is the only one that is predictable, debuggable, and respects both the user's input and the call's audit trail.

**The correct fallback is a documented eviction order.** The other three options each fail in a specific, predictable way. ### Why reject (option 0) is usually wrong Overflow is mostly the system's responsibility, not the user's. Retrieval over-fetched. Summary grew. Tool results accumulated. The user's literal input is rarely the overflow cause; even when it is, the system can apply eviction to the surrounding context and still serve the request. Hard rejection is the right move only when the user's own input alone exceeds the budget after everything else has been evicted. That is a real failure mode and a clear error is the right response, but it should be the last resort, not the first. ### Why silent head-truncation (option 1) is catastrophic The start of the prompt holds the system prompt: persona, safety policies, tool definitions, prompt cache friendly stable content. Truncating from the start kills the agent's identity and safety guarantees. Production failure mode: agents start violating brand voice or safety policy under high-context load, and the failures only show up at the budget edge. Most naive implementations default to head-truncation because string slicing from the start is the cheapest possible operation. The cost shows up later, when the agent starts misbehaving and the team has to figure out why. ### Why silent model-swap (option 3) is wrong A larger-context model is a different model. It has different pricing (often 3-5x), different behavior characteristics (some are slower, some are weaker on reasoning, some have different safety calibration), and different rate limits. Swapping silently means the caller pays more, gets different behavior, and cannot audit what happened. Model selection is a policy decision; a per-call automatic fallback is operating outside that policy. The right pattern is to declare a model ladder explicitly.