Walk through BAML, Outlines, OpenAI Structured Outputs, and Pydantic with retry as paths to structured output. For each, name the layer it operates at, the guarantee it provides, and the failure mode it leaves open.
Free · 2 AI evals / day
Structured output is the problem of getting an LLM to return data that matches a specific schema, like a JSON object with named fields and types. Four common tools take very different approaches to the problem.
**BAML** is an application-layer tool. You define a typed function in BAML's DSL, codegen produces a client in Python or TypeScript. The client calls the model, parses the response, and retries on parse failure. You get type-safety at the call site and eventual valid output with bounded retries. The failure mode is that a pathological prompt can exhaust the retry budget without ever parsing.
**Outlines** is a decoding-layer tool. It derives a finite-state machine from your schema and masks the next-token logits during sampling so that invalid tokens have zero probability. Invalid output cannot be produced at all. The failure mode is that this only works on models where you control the sampling, usually self-hosted models, and the schema can degrade quality if it forces tokens the model would not naturally pick.
**OpenAI Structured Outputs** is provider-side constrained sampling. Same kind of guarantee as Outlines but only available on OpenAI endpoints that opt in. The failure mode is vendor lock and the limited model list.
**Pydantic + retry** validates the model's output after it is generated. If validation fails, you re-prompt with the error message. The guarantee is the weakest: you just get told if it is wrong, you do not prevent it.
The four approaches live at three different points in the LLM call lifecycle: before the first token (codegen), during sampling (decoding), and after the response (validation). Anchor each tool to a point and the comparison clicks.
**BAML** operates at the application layer with codegen. You declare a typed function in `.baml` source files; the toolchain generates a typed client at build time. The client templates the prompt, calls the model, parses the response, and runs bounded retries on parse failure. Guarantee: type-safety at the call site plus eventual valid output within the retry budget. Failure mode: adversarial inputs exhaust the budget and surface as exceptions.
**Outlines** operates at the decoding layer. It derives a finite-state machine from your schema and masks the next-token logits during sampling so invalid tokens get zero probability. Guarantee: invalid output cannot be produced. The strongest of the four. Failure modes: requires sampling control (mostly self-hosted models), can degrade quality if the schema forces low-probability tokens, FSM construction has overhead on complex schemas.
**OpenAI Structured Outputs** is provider-side constrained sampling on supported models (`gpt-4o-2024-08-06` and later). Same guarantee as Outlines on the supported endpoints. Failure modes: vendor lock-in and a limited model list.
**Pydantic + retry** is the weakest. Let the model generate freely, validate against a Pydantic model, re-prompt with the error on failure. Guarantee: detection only. Failure modes: silent partial validation (schema valid, values wrong) and unbounded retry cost when the model never converges.
The four are not substitutes. They sit on a prevent versus detect axis; pick by provider, cost, and certainty needs.
Quick reference
TL;DR
The four sit on a prevent versus detect spectrum: Outlines and OpenAI Structured Outputs prevent invalid output at sampling; BAML retries at the call site; Pydantic only detects after.
Imagine four ways to make sure a kid only colors inside the lines. The strongest way is to print the page with a raised plastic border so the crayon physically cannot cross it. The next strongest is to print only with crayons that turn invisible outside the lines. The middle way is to let the kid color freely, then before showing the picture to anyone, check whether they stayed inside and ask for a redo if not. The weakest way is to show the picture without checking and tell the kid afterward whether they did it right. All four end up with a coloring page; only the first two make the wrong outcome impossible.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Schema with recursive types or complex regex patterns that the decoding-layer FSM cannot represent Outlines and provider-side prevention have schema complexity limits; application-layer parsing handles arbitrary schemas
Tasks where free-form prose quality matters more than schema strictness Decoding-layer prevention can degrade quality by forcing tokens the model would not naturally pick
Prototype phase where the schema itself is in flux Codegen and FSM construction add iteration friction when the schema changes daily; Pydantic plus retry is faster to iterate
Quick scan · 2–4 min
Everything important, quickly.
7 minutes: the lifecycle-anchored framing, the four tools mapped to it, production tradeoffs, and the stacking pattern.
Side-by-side
Tool
Layer
Guarantee
Primary failure mode
Outlines
Decoding (FSM logit mask)
Cannot produce invalid output
Needs sampling control; quality may degrade on restrictive schemas
OpenAI Structured Outputs
Decoding (provider-side)
Cannot produce invalid output on supported models
Vendor lock-in; limited to specific OpenAI models
BAML
Application (codegen + retry)
Eventual valid output within retry budget
Pathological inputs exhaust the budget
Pydantic + retry
Application (validate after)
Will detect invalid output; will not prevent it
Silent partial-validation; unbounded retry cost
Real-world examples
Real products, models, and research that use this idea.
1OpenAI Structured Outputs was introduced on gpt-4o-2024-08-06 and by 2026 is standard across gpt-5.5 and the o5 reasoning family.
2BAML is used by Notion-style and Vercel-style teams for schema-first typed LLM functions across TS and Python.
3Outlines ships as part of the vLLM ecosystem and is the standard for self-hosted constrained generation.
4Pydantic plus retry remains the most common pattern in early-stage prototypes and in pipelines where the schema is loose.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does Outlines handle a JSON schema with a recursive type?
Sign in or go Pro to see the approach for this follow-up.
QWhat is the right retry budget for Pydantic plus retry, and how do you tune it?
Sign in or go Pro to see the approach for this follow-up.
QWhen would you stack BAML on top of OpenAI Structured Outputs?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Treating Pydantic plus retry and OpenAI Structured Outputs as equivalent. They are at opposite ends of the prevent vs detect spectrum.