Where should tool/function call schemas live in the assembled context, and why?

Tool schemas are the dictionary the model uses to decide which action is available and how to call it. They should live in the system prompt or a dedicated tool-definitions block at the very top of the assembled context. Two concrete reasons for top placement. First, **prompt caching only works on stable prefixes**. Every major provider caches the prompt prefix up to the first byte that changes between calls. If the 2400-token tool definition block is at the top and identical every call, the provider serves it from cache at roughly 10% of the normal input cost. If it is in the middle, anything before it that changes per call invalidates the cache. Top placement makes the schemas free after the first call. Second, **anchoring**. The model reads the rules of the world before it reads the user's task. The available actions, their argument shapes, and how to use them are loaded into the model's working state before any user content. That improves tool-call accuracy because the model is not learning the action space mid-response. The assumption behind this placement is that the schemas do not change per call. If a per-user feature flag enables or disables certain tools, that flag goes into a small separate per-call block after the cached prefix, so the bulk of the schema definitions stay cached and only the small diff is paid each call.

**Tool schemas belong in the system or developer block at the very top of the context.** Two production reasons make this near-universal in 2026 agent stacks. ### Reason 1: prompt caching only pays off on stable prefixes Anthropic Claude prompt caching, OpenAI's automatic prefix caching, and Gemini's context caching all share the same property: they cache the contiguous prefix up to the first byte that differs between calls. Eight tool schemas at 300 tokens each is 2,400 tokens. If the block is identical every call, the provider serves it from cache at 10-15% of normal input price (Anthropic's pricing is roughly 1/10 of input for cached reads). Drop a per-user variable into the middle of the block and you wipe out the cache hit for everything after that variable. The provider has no way to know that the rest is identical; it just sees a divergence and pays full price for everything downstream. ### Reason 2: anchoring the action space The model reads the schemas before it reads the user task. That order matters because the model is building up its working state about what is even possible before it sees what to do. Tool-call accuracy in agent eval benchmarks improves measurably when schemas precede the task versus when they trail it, because the model is not learning the action space inside its first response. ### The placement assumes static schemas If a per-user or per-session flag changes which tools are available, do not embed that flag inside the schema block. Move it to a small per-call section after the cached prefix: 'For this user, tools X and Y are disabled.' That keeps the bulk of the schema definitions cached and only pays for the few-token diff per call. If an entire tool definition itself varies per user (rare; usually a sign of a deeper design problem), split the schema block into a stable shared half and a variable per-user half, with the variable half placed at the end of the system block so the cache still covers the shared portion.