In a production prompt for generating a single SQL query, what's the right way to make sure the model returns only the SQL without trailing commentary?
Suppressing trailing commentary on a generated SQL query is a layered problem, not a single-knob problem. Three controls work together.
**Stop sequence.** This is the strongest tool. You tell the API that when the model emits a specific byte sequence (a closing fence like ```` ``` ````, or a sentinel like `END_SQL`), generation stops immediately at that point. The model literally does not get to sample any tokens past the sequence. In your few-shot examples, demonstrate the pattern: the SQL ends with the terminator on its own line. The model learns to emit the terminator after the SQL, and the API's stop logic cuts the response there.
**max_tokens.** This caps the worst case. Size it for a realistic SQL length budget (say 500 tokens for a complex analytical query). It is a guardrail, not a primary control. If you size it tight enough to cut off commentary, you risk truncating long but legitimate SQL. If you size it loose enough to fit any SQL, it does not suppress commentary.
**Post-hoc parser.** The safety net. Take the response, strip whitespace, parse out everything up to the first occurrence of the terminator (or until you have a complete SQL statement). This catches the edge cases where the model added leading commentary, did not emit the terminator, or wrapped the SQL in extra prose.
The instruction in the system prompt (return only the SQL, no explanation) is necessary too. It is what makes the model prefer the terminator and avoid commentary in the first place. But on its own it is not a guarantee.
Production output control is layered because each layer fails differently and the layers compose.
**Layer 1, stop sequence.** The strongest control. Specifying a stop sequence (`stop_sequences=['```']` on Anthropic, `stop=['```']` on OpenAI) terminates generation at the byte boundary, so commentary never gets sampled. Pair it with a few-shot example where the assistant emits the terminator after the SQL, which trains the pattern to be self-anchoring: the model wants to emit the terminator because that is what the demonstrations show.
**Layer 2, max_tokens.** The cost and latency guardrail. Size it for a generous but realistic SQL length (200 to 500 tokens for analytical queries, 50 to 100 for simple lookups). It exists to bound the worst case if everything else fails. It is not a primary control because the only way to make it suppress commentary is to truncate; that hurts legitimate long outputs.
**Layer 3, post-hoc parser.** The edge-case net. Strip whitespace, take content up to the first terminator (or up to the first complete statement). This catches the cases where the model added leading prose ('Here is the SQL:'), did not emit the terminator, or wrapped the output in extra formatting.
The system-prompt instruction ('return only the SQL, no commentary') is the orientation layer that makes the model prefer the right shape. It is necessary but not sufficient on its own, because production has long tails.
The failure-mode mapping: instruction alone leaks on one in twenty edge cases; stop sequence alone fails when the model never emits the sentinel; max_tokens alone either truncates or does nothing; parser alone is brittle if the model output is highly variable. The four together compose into a reliable extractor.
Quick reference
TL;DR
Use a stop sequence as the primary control, max_tokens as the cost and latency guardrail, and a post-hoc parser as the edge-case net. Each layer catches what the others miss.
Imagine you asked a chatty friend to write you a recipe and only the recipe. Three things keep them honest. First, you write 'END' at the bottom of the page and tell them to stop writing there; that is the stop sequence. Second, you only hand them a small piece of paper; that is max_tokens. Third, when they hand it back, you read only up to where it says END and ignore anything they scribbled after; that is the post-hoc parser. Any one alone fails sometimes. All three together always give you just the recipe.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
The model needs to return SQL plus explanation in the same response Stop sequence will cut off the explanation; switch to structured output (JSON with sql and explanation fields) so both arrive together.
Your downstream system already runs a SQL parser on the output If sqlparse or sqlglot is the source of truth, the parser layer alone may be sufficient; the stop sequence is then a latency optimization, not a correctness control.
The legitimate output may genuinely contain triple backticks or your chosen sentinel Inside-string-literal collisions cause premature termination; pick a different terminator (a longer multi-line sentinel) or escape the conflict in few-shot demonstrations.
Quick scan · 2–4 min
Everything important, quickly.
3 min: stop sequence as primary control + max_tokens as guardrail + post-hoc parser as net + system-prompt as orientation + how the layers compose.
Real-world examples
Real products, models, and research that use this idea.
1Anthropic's prompt-engineering cookbook explicitly recommends fenced code blocks with the closing fence as a stop sequence for Claude Opus 4.7 SQL and code generation, paired with one or two few-shot examples.
2OpenAI's structured outputs documentation shows the same fence plus stop pattern for GPT-5.5 code generation, alongside JSON mode for cases that need schema siblings.
3Vanna AI and other text-to-SQL production stacks running on Claude Sonnet 4.x and GPT-5.5 ship the three-layer pattern: stop sequence on the closing fence, max_tokens at the 99th percentile of historical SQL length, plus a sqlparse-based extractor as the parser layer.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you pick a stop sequence that will not collide with legitimate output?
Sign in or go Pro to see the approach for this follow-up.
QWhen would JSON mode beat the fence plus stop pattern?
Sign in or go Pro to see the approach for this follow-up.
QHow does this pattern interact with streaming?
Sign in or go Pro to see the approach for this follow-up.
QWhat does the parser fall back to when the model never emits the terminator?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Relying on a system-prompt instruction alone to suppress commentary, then catching a regression weeks later when one in twenty SQL outputs ships with a trailing 'Note: this assumes X' that breaks the downstream parser.