Structured output from an LLM is rarely a one size fits all problem. Five common patterns each fit a different shape of production constraint.
Provider-native structured output (OpenAI's response_format, Anthropic's tool-use, OpenAI Structured Outputs with strict JSON schema) is the simplest integration. You hand the provider a schema and the provider enforces it. The cost is provider lock-in and accepting whatever level of strictness they give you.
Pydantic plus Instructor in Python, or Zod plus runtime validation in TypeScript, both work by validating the model's output after the fact and retrying on failure. They fit naturally when the rest of your codebase already speaks Pydantic or Zod.
A hand-rolled JSON Schema plus a repair loop is the most portable choice. You define the schema, validate, and prompt the model to fix any errors. It works with any LLM but you write the loop yourself.
Grammar-constrained decoding (Outlines, llama.cpp grammars, Guidance) is the strictest. The decoder physically cannot emit invalid tokens. You pay 10 to 20 percent latency and you usually need a local model, but the output is guaranteed valid.
Match the pattern to your ecosystem, your tolerance for retries, and how much you can spend on latency.
Schema validation for LLM outputs splits into five common patterns. The right choice depends on two axes: language ecosystem and strictness budget.
`Pydantic + Instructor` fits Python backends that already speak Pydantic. Instructor wraps the LLM call, parses the response into the Pydantic model, and retries on validation failure. You get type-safe outputs at the cost of one possible retry per call.
`Zod + TypeScript` is the symmetric fit for Node and Next.js codebases. The rest of the type system is Zod-driven, and runtime guards line up with compile-time types.
A hand-rolled `JSON Schema + repair loop` is the provider-agnostic choice. You write the validate and retry loop yourself; it works with any LLM that accepts a JSON hint. The cost is the loop code and the occasional retry. The benefit is portability across providers.
Provider-native structured-output mode (OpenAI `response_format` with strict JSON schema, Anthropic tool-use, Gemini structured output) is the simplest integration. You hand the provider a schema, the provider's decoder enforces it, and you avoid the loop. The cost is provider lock-in and whatever strictness ceiling they offer.
Grammar-constrained decoding (Outlines, llama.cpp grammars, Guidance) is the strictness ceiling. The decoder masks logits to physically prevent invalid tokens. Output is guaranteed valid against the grammar. The cost is 10 to 20 percent latency, usually a local model, and a grammar definition. Pick it when downstream consumers (embedded systems, deterministic pipelines) cannot tolerate any malformed output.
Quick reference
TL;DR
Pick the validator by ecosystem first (Pydantic for Python, Zod for TS) and by strictness budget second (provider-native is easy, grammar-constrained is bulletproof).
Imagine you ordered a pizza and you only want round pizzas with cheese. Five restaurants offer five different guarantees. One says trust us, we'll send a round cheese pizza. One says we'll send something, you check it, and if it's wrong call us back for a retry. One says we built a machine that physically cannot produce anything except round cheese pizzas; it costs more time but you never get a bad pie. One says we use Python so we'll Pydantic-check it. One says we use TypeScript so we'll Zod-check it. Schema validation for LLMs is the same menu. You match the guarantee to how much pain a wrong order causes downstream, then you match the validator to whatever language your kitchen already speaks.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Prototype where speed of iteration matters more than guarantees Heavy validation infrastructure slows iteration; a permissive JSON parse with manual checks is fine until the prototype goes to production.
Truly free-form output where the schema would be arbitrary Forcing a schema on a creative-writing task or open-ended summarization adds friction without benefit; let the output be free text.
Routes where the LLM output is consumed by another LLM that tolerates noise If the downstream consumer is another model that can reason past minor format slips, strict validation is overhead the pipeline does not need.
Quick scan · 2–4 min
Everything important, quickly.
3 min: five patterns mapped to ecosystem and strictness axes, when each pattern earns its cost, and how to avoid the over-engineering default of grammar-constrained everything.
Real-world examples
Real products, models, and research that use this idea.
1Anthropic tool-use and OpenAI Structured Outputs (strict JSON schema) cover the simplest provider-native integrations in production today.
2Instructor on FastAPI services is the dominant Python pattern for type-safe LLM output in 2025.
3Outlines and llama.cpp grammars ship grammar-constrained decoding for local-model deployments where invalid output cannot be retried.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you handle schema migration when the LLM output shape needs to change?
Sign in or go Pro to see the approach for this follow-up.
QWhat is the right retry policy when validation fails?
Sign in or go Pro to see the approach for this follow-up.
QWhere does grammar-constrained decoding hit a wall in production?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Picking the most strict tool by default (grammar-constrained decoding everywhere) instead of matching cost and strictness to actual downstream requirements.