Design the structural layout for a production prompt powering a medical document Q&A system over retrieved clinician notes.

Short answer·Hard·4.0 · 0·~3 min·Asked atPaytm Servicenow Typeface·Relevant atAnthropic OpenAI

Attempt it

You're building a medical document Q&A system: clinicians ask questions, your app retrieves relevant patient note chunks, and the LLM answers based on those chunks. Specify the structural layout of the prompt: what each component is, where it lives (system vs user), and why. Address: instructions, output format, few-shot examples, refusal directive, citation constraint, context layout. Be specific about decisions and tradeoffs.

Free · 2 AI evals / day

The structure has two halves and the split is the point. The **system message** carries the parts that do not change between calls. Role: a medical document Q&A assistant for clinicians; answer only from the provided notes. Output format: a fixed JSON (structured text format) schema with an answer field, a citations array, and a confidence enum (a fixed list of levels). Refusal rule: if the notes are not enough, set confidence to insufficient and explain what is missing. Citation rule: every factual claim must be backed by a verbatim quote from a specific note. Two or three few-shot examples show the JSON shape on a normal case, an insufficient-info case, and an ambiguous-symptom case. The **user message** carries the parts that change every call. The retrieved patient note chunks come first, wrapped in XML tags with note ids and dates. The clinician's question comes at the very end, so it sits in the strongest attention position. This split makes the system message cacheable (cheaper, faster), keeps the dynamic content where attention is strongest, and makes citations checkable downstream.

The split is system vs user. It is shaped by three forces: caching, attention bias, and downstream evaluability. System message (stable, cacheable). Role and scope: medical document Q&A; answer only from the provided notes. Output format: a JSON schema with answer, citations (each a `{note_id, quote}` pair), and confidence as an enum (high, medium, low, insufficient). Refusal directive: when notes are insufficient, set confidence=insufficient and describe what is missing. Citation contract: every claim must be supported by a verbatim quote from a specific note; paraphrased citations are not allowed. Two or three few-shot examples cover normal, insufficient-info, and ambiguous cases. User message (dynamic). Retrieved patient note chunks are wrapped in XML tags with stable ids and dates: `<patient_notes><note id="n1" date="2024-03-15">...</note></patient_notes>`. The clinician's question follows. It sits in `<question>...</question>` at the very end. Stable system enables prompt caching. On Anthropic and OpenAI hosted models this saves 50 to over 90 percent of prefix cost when few-shots are long. The dynamic tail benefits from head and tail position bias documented in Lost in the Middle. Verbatim quotes plus stable note ids make citation correctness checkable in regression eval; paraphrased citations are not. The explicit confidence enum gives downstream code a routable signal: low-confidence answers can be sent to human review.

TL;DR

Put role, format schema, refusal rule, citation contract, and few-shot examples in the cacheable system message; put XML-tagged retrieved notes and the clinician's question, in that order, in the user message.

Memory aid

Easy to grasp

Imagine you are a brand-new hospital intern. Your supervisor hands you a binder of rules every morning. Then, for each patient case, they slide a stack of that patient's notes across your desk and ask one question. The binder never changes day to day. It tells you to answer only from the notes, write your answer in a fixed form, and admit when the notes do not say enough. The stack of notes is different every case. The question always comes after the notes, so it is the last thing you read before you answer. That two-part split (rules in the binder, evidence and question on the desk) is how this prompt is laid out.

Key concepts

Structured Output RAG

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Medical document Q&A is the kind of task where prompt design either makes or breaks the product. The stakes are high (hallucinated medical claims are unacceptable). The inputs are messy (notes from many encounters, different formats, partial coverage). The downstream consumers are humans who will act on the answer. Every structural choice in the prompt has to be defended.

Why split system from user at all

The system vs user split is a human convention with three real production payoffs.

The first is prompt caching. Anthropic lets you mark a cache breakpoint after the system block with cache_control. OpenAI caches matching prefixes automatically, and newer OpenAI models also accept explicit breakpoints. Everything before the breakpoint is reused across calls; only the dynamic tail is re-processed. For a medical Q&A prompt with two or three few-shot examples, the system block can be 4 to 8k tokens. Caching that turns the per call cost into mostly the user-side payload, which is typically the notes plus the question. The savings on the cached prefix are 50 to over 90 percent, depending on provider, model and traffic patterns.

The second is attention placement. Long-context LLMs use information at the start and end of the input most reliably (the U-shaped curve in Lost in the Middle). The system block sits at the start; the question sits at the end of the user message. Both are in the strongest positions. The retrieved notes sit in the middle, the weakest position. That is a cost of the layout, accepted because caching needs the stable block first and the question works best last. To reduce it, keep the retrieved set short and put the most relevant notes nearest the question.

The third is eval discipline. The system block is what you author and version. It runs through regression CI on every change. The user block is what flows in at request time; it runs through retrieval-side monitoring. Each block has its own process, tooling, and alerts. Keeping the split clean keeps the engineering organization clean.

The output schema and the confidence enum

Citation contract and refusal path

Dynamic layout, few-shot examples, and production tooling

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

With Anthropic prompt caching, a clinical Q&A system can cache the stable system block (schema + few-shot) so that only the patient notes and question are processed at full price on each call.
OpenAI Structured Outputs with a strict JSON schema (unlike plain JSON mode, which only guarantees valid JSON) can enforce an explicit confidence enum, so downstream code can route low-confidence answers to human review.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you regression-test this prompt before shipping a change?

Golden set of (notes, question, ideal JSON) triples; metrics for faithfulness (verbatim citation match), format compliance, and refusal correctness; block merge on regressions past noise floor.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Putting retrieved patient notes in the system message because they look like context, which breaks prompt caching and loses the recency-attention boost that the clinician's question needs.

60 second bullets to scan on the way to the call.

The system vs user boundary and why it matches caching
The output schema with confidence enum

Primary sources. Browse if you want the original framing.