You're building a medical-document Q&A system: clinicians ask questions, your app retrieves relevant patient-note chunks, and the LLM answers based on those chunks. Specify the structural layout of the prompt: what each component is, where it lives (system vs user), and why. Address: instructions, output format, few-shot examples, refusal directive, citation constraint, context layout. Be specific about decisions and tradeoffs.
Free · 2 AI evals / day
The structure has two halves and the split is the point.
The **system message** carries the parts that do not change between calls. Role: a medical-document Q&A assistant for clinicians; answer only from the provided notes. Output format: a fixed JSON schema with an answer field, a citations array, and a confidence enum. Refusal rule: if the notes are not enough, set confidence to insufficient and explain what is missing. Citation rule: every factual claim must be backed by a verbatim quote from a specific note. Two or three few-shot examples that demonstrate the JSON shape on a normal case, an insufficient-info case, and an ambiguous-symptom case.
The **user message** carries the parts that change every call. The retrieved patient-note chunks wrapped in XML tags with note ids and dates, and the clinician's question at the very end so it sits in the strongest attention position.
This split makes the system message cacheable (cheaper, faster), keeps the dynamic content where attention is strongest, and makes citations checkable downstream.
The split is system vs user. It is shaped by three forces: caching, attention bias, and downstream evaluability.
System message (stable, cacheable). Role and scope: medical-document Q&A; answer only from the provided notes. Output format: a JSON schema with answer, citations (each a `{note_id, quote}` pair), and confidence as an enum (high, medium, low, insufficient). Refusal directive: when notes are insufficient, set confidence=insufficient and describe what is missing. Citation contract: every claim must be supported by a verbatim quote from a specific note; paraphrased citations are not allowed. Two or three few-shot examples cover normal, insufficient-info, and ambiguous cases.
User message (dynamic). Retrieved patient-note chunks wrap in XML tags with stable ids and dates: `<patient_notes><note id="n1" date="2024-03-15">...</note></patient_notes>`. The clinician's question follows. It sits in `<question>...</question>` at the very end.
Why this layout works. Stable system enables prompt caching. On Anthropic and OpenAI hosted models this saves 50 to 90 percent of prefix cost when few-shots are long. The dynamic tail benefits from head and tail position bias documented in Lost in the Middle. Verbatim quotes plus stable note ids make citation correctness checkable in regression eval; paraphrased citations are not. The explicit confidence enum gives downstream code a routable signal: low-confidence answers can be sent to human review.
Quick reference
TL;DR
Put role, format schema, refusal rule, citation contract, and few-shot examples in the cacheable system message; put XML-tagged retrieved notes and the clinician's question, in that order, in the user message.
Imagine you are a brand-new hospital intern, and your supervisor hands you a binder of rules every morning, then on each patient case slides a stack of that patient's notes across your desk and asks one question. The binder never changes day to day. It tells you to answer only from the notes, write your answer in a fixed form, and admit when the notes do not say enough. The stack of notes is different every case, and the question always comes after the notes so it is the last thing you read before you answer. That two-part split (rules in the binder, evidence and question on the desk) is exactly how this prompt is laid out.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Single-turn lookups where the notes fit in a few hundred tokens and there is no caching benefit The cacheability boundary stops paying off; a flatter prompt is simpler and the production wins are negligible.
Open-ended clinical research where verbatim grounding is too restrictive Research questions sometimes need synthesis across sources; a verbatim-only contract is the wrong fit for that task and should be relaxed deliberately.
Workflows that consume freeform prose downstream and cannot parse JSON Structured output is overkill when no downstream system needs the structure; prose output with inline citations may be a better fit.
Quick scan · 2–4 min
Everything important, quickly.
5 min: system vs user split + cacheability boundary + verbatim citation contract + refusal path with confidence enum + XML-tagged notes plus end-placed question + regression-eval hooks.
Real-world examples
Real products, models, and research that use this idea.
1Anthropic Claude Opus 4.7 with prompt caching is widely used for clinical decision support pilots where the stable system block (schema + few-shot) is cached and only the patient notes and question vary per call.
2OpenAI GPT-5.5 in JSON-mode is used by RAG vendors to enforce structured outputs with explicit confidence fields, with downstream code routing low-confidence answers to human review.
3RAGAS and Anthropic's faithfulness-judge prompts power regression CI for grounded-QA pipelines, gating prompt changes on the verbatim-citation contract.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you regression-test this prompt before shipping a change?
Sign in or go Pro to see the approach for this follow-up.
QWhat changes if a single patient has dozens of notes that exceed the context window?
Sign in or go Pro to see the approach for this follow-up.
QHow would you handle multi-turn follow-up questions on the same patient?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Putting retrieved patient notes in the system message because they look like context, which breaks prompt caching and loses the recency-attention boost that the clinician's question needs.