Which of the following content types belong in the SYSTEM message (not the user message) in a production chat style prompt?
Anything stable across the session belongs in system; anything that changes per turn (the user's question, retrieved RAG chunks) belongs in user.
Picture two folders on a clipboard for a help-desk worker. The top folder is the rulebook: who they work for, how to talk, what to refuse, what format their replies take. They open it once a day and follow it on every call. The bottom folder gets a new sheet of paper every time the phone rings: today's caller name, today's question, the printout the system just pulled about that caller's account. The rulebook is the system message. The fresh sheets are the user message. Mixing them, taping today's caller note inside the rulebook: breaks the system the next time the phone rings.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
This question is a deceptively concrete one. On the surface, the candidate has to sort six items into the two buckets. Underneath, the placement decisions are driven by the same architectural property every chat-style LLM API shares: system is the stable conditioning prefix, user is the dynamic per-turn surface. The deeper a candidate goes on why each item lands where it does, the more senior they sound.
The deep dive below walks through the rule, the two non-obvious cases (RAG context and personalization), the 2026 cost story that prompt caching introduces, and the failure modes that show up at scale when teams blur the line.
The persistent vs per-turn rule
Sort each candidate content item by asking one question: does this survive unchanged to the next API call in this session? If yes, system. If no, user.
Role definition, output format, refusal behavior, and tone all survive. They are properties of the agent itself; they should be true on call 1, call 10, and call 1,000 of the session. Putting them in system means you write them once, the model conditions on them every turn, and you do not pay for resending them in raw form (prompt caching reuses the prefix).
The current user question obviously does not survive; it is replaced on every turn. Retrieved RAG context does not survive either, even though it feels more like 'background' than 'a question'. The retriever runs against each query and returns different chunks every time. Locking those chunks into system would force you to rewrite system on every call, which is the exact thing system was designed not to do.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Anthropic's prompt-engineering guide explicitly recommends user-message placement for retrieved RAG chunks; Claude Opus 4.7 attends better to context placed close to the query.
- OpenAI's structured-outputs and JSON-mode guidance puts the schema directive in system (stable) and the data to extract in user (dynamic).
What an interviewer would ask next. Try answering before peeking at the approach.
QSuppose you must personalize the system block per user. How do you preserve a high cache hit rate?
Keep the truly stable prefix at the top (persona, rules, format), push per-user data to the end or into user. Mention Anthropic cache-control breakpoints and how they let you cache the shared prefix and skip the personalized tail.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Putting retrieved RAG chunks into the system message because they 'feel like context'; they change every query, so they belong in user near the question.
60 second bullets to scan on the way to the call.
The persistent vs per-turn rule of thumb
Why retrieved RAG context belongs in user, not system
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.