In a few-shot prompt that extracts structured fields from invoices, why is a distinctive stop sequence (like a custom END_INVOICE marker) better than a paragraph break or the natural end of response?
Short answer·Medium·4.0 · 0·~3 min·Relevant atAnthropicOpenAI
You're an LLM engineer designing a few-shot prompt that extracts structured fields (vendor, total, line items) from invoice text. You're picking how the model should signal the end of each extracted record. Walk through why a distinctive stop sequence (like a custom END_INVOICE marker) is better than a paragraph break or relying on the model's natural end of response, and what properties make a stop sequence robust.
Free · 2 AI evals / day
A stop sequence is what tells the API to stop generating tokens at a specific byte boundary. For structured extraction (invoice fields, JSON records, code blocks) a distinctive marker like `END_INVOICE` is much safer than relying on a paragraph break or whatever the model produces naturally.
**Reason one: paragraph breaks appear inside invoices.** Real invoice text has blank lines between addresses, between line items, between the header and the totals block. If you use a blank line as the terminator, generation cuts off the moment the model produces the first internal blank line, mangling the record. A custom marker that is structurally impossible inside real invoice text avoids this.
**Reason two: few-shot examples train the model to emit the marker.** If every example in your prompt ends with `END_INVOICE`, the model learns that the right shape is to emit `END_INVOICE` after the fields. The marker becomes self-anchoring: the model wants to produce it because the examples show it.
**Reason three: it decouples the termination contract from the format.** Adding a new field later (a payment_terms line) does not change the terminator. Format-specific terminators (a closing JSON brace, an end of paragraph) are tied to the format and have to be revisited every time the format changes.
**Reason four: it composes across tasks.** If your prompt extracts an invoice and then a summary, distinct terminators (`END_INVOICE` and `END_SUMMARY`) make the boundaries unambiguous. Reusing one terminator across tasks creates ambiguity.
Finally, the marker on its own does nothing unless you pass it to the API's `stop_sequences` parameter. The API parameter is what cuts generation at the boundary. The marker design is what trains the model to emit the boundary in the first place. Both together is the working pattern.
A distinctive stop sequence wins on four properties: collision safety, self-anchoring through few-shot, decoupling from the format, and composability across tasks.
**Collision safety.** Paragraph breaks routinely appear inside invoice content (addresses, line items, descriptions). A blank line as the terminator means the first internal break ends generation prematurely. A multi-character marker like `END_INVOICE` or `</invoice>` is structurally impossible inside real text.
**Self-anchoring.** When every few-shot example emits the terminator after the fields, the model learns the shape: extract fields, then emit the marker. The terminator becomes part of the output the model wants to produce, which gives you training-side reinforcement on top of the API-side stop.
**Decoupling from the format.** A custom terminator does not change when you add a field. The extraction can grow with new fields (payment_terms, currency, billing_period) while the termination contract stays stable. Format-coupled terminators have to be revisited every time the format changes.
**Composability.** Distinct terminators per output type (`END_INVOICE`, `END_LINE_ITEM`, `END_SUMMARY`) let you compose prompts that handle multiple records without ambiguity.
**The implementation pairs three pieces.** The marker design (what the sentinel looks like). The few-shot demonstration (every example ends with it). The API control (pass the marker to `stop_sequences` on Anthropic or `stop` on OpenAI). The marker alone does not enforce anything; the API alone does not train the model to emit it.
**The natural-end alternative fails three ways.** The model adds trailing commentary. `max_tokens` truncates mid-record. The model rambles when unsure about a field. A distinctive stop sequence eliminates all three because the API cuts generation at the boundary regardless.
Quick reference
TL;DR
A marker like END_INVOICE is collision-safe, self-anchoring through few-shot demos, decoupled from format changes, and composable across tasks. Pair it with the API's stop_sequences for byte-level enforcement.
Imagine you are training a friend to read invoices and write down what they see. You could tell them 'stop when you reach a blank line' but invoices have blank lines inside them. Instead you say 'when you are done with each invoice, write THE END on its own line'. Now you have a clear signal that will not be confused with anything inside the invoice. You also show them ten example invoices that all end with THE END, so they pick up the habit. And you tell the machine reading their notes to ignore anything after THE END. Three layers, one clean signal.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
The output is a single short string with no structure or commentary risk If the response is something like a yes/no classification, max_tokens=1 or a constrained-decoding option suffices; stop sequences add complexity without benefit.
The downstream consumer wants JSON with typed fields Structured-output mode (Anthropic tool_use, OpenAI strict JSON) gives type guarantees and schema validation that string-terminator patterns cannot match.
The legitimate output may contain any reasonable sentinel you pick If your output domain is unbounded (free-form documents that could contain anything), use structural enforcement (JSON schema, regex on field shapes) rather than a string sentinel.
Quick scan · 2–4 min
Everything important, quickly.
3 min: four properties of a good terminator (collision-safe, self-anchoring, decoupled, composable) + the three-layer contract (few-shot, model, API) + parser as safety net.
Real-world examples
Real products, models, and research that use this idea.
1Anthropic's prompt-engineering cookbook recommends XML-style closing tags as stop sequences for structured extraction with Claude Opus 4.7, with the tag demonstrated in every few-shot example.
2OpenAI's GPT-5.5 vision-extraction examples for invoices and receipts pair a distinctive END_RECORD sentinel with the stop parameter and few-shot demonstrations of the terminator.
3Stripe's document-extraction pipelines for KYC and receipt processing run on Claude Sonnet 4.x and Llama 4 finetunes with distinct terminators per record type, enabling the downstream parser to split concatenated outputs unambiguously.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you handle a case where the model genuinely needs to emit content containing your terminator?
Sign in or go Pro to see the approach for this follow-up.
QWhat is the parser layer's job when the model fails to emit the terminator?
Sign in or go Pro to see the approach for this follow-up.
QWhen does structured-output mode (Anthropic tool_use, OpenAI strict JSON) beat the stop-sequence pattern?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Using a paragraph break as the terminator on a structured-extraction prompt, then watching extractions get truncated whenever the model produces a blank line inside a multi-line address or line-item block.