Which of the following content types belong in the SYSTEM message (not the user message) in a production chat-style prompt?

The four correct answers are role definition, output-format directive, refusal directive, and tone/style. The two wrong answers are the user's current question and retrieved RAG context. The rule is simple: **persistent vs per-turn**. The system message is the model's standing orders. It carries things that should be true on every call in the session: who the model is, how it should answer, what format to use, what it must refuse. You set it once and forget about it. The user message is what changes each time. The current question is obviously per-turn. Retrieved RAG context is per-turn too, even though it can feel like 'background': the retriever pulls different chunks for every query. If you put those chunks in system, you would have to rewrite the system block on every call, which defeats the purpose of having a stable system. When in doubt, ask: does this content survive to the next call unchanged? If yes, system. If no, user.

Four belong in system: role definition, output-format directive, refusal-behavior directive, and tone/style. They are persistent rules the model should follow across every turn in this session, set once and reused. Two belong in user. The user's current question is the per-turn input; it has no other home. Retrieved RAG context goes in user too, and this trips people up. The chunks change every query, so they cannot be cached as a stable system prefix. Modern guidance from both Anthropic and OpenAI recommends placing retrieval results close to the query to win the recency-attention bias. The discipline that follows: keep the system block stable and big, put dynamic per-turn data in user. In 2026 this also has a direct cost consequence. Prompt caching gives a stable system prefix roughly a tenth of normal input cost on cache hits. Every dynamic byte you put in system is a byte that breaks the cache and pays full price.