Design a validate and repair loop for an endpoint that must return valid JSON
An endpoint must always return schema valid JSON to a downstream system. Even with the provider's JSON mode enabled, describe the validate and repair design you'd put around the model call.
JSON mode guarantees syntax, not your schema, so always validate the parsed object yourself. On failure, repair-parse trivia or re-ask with the exact error, cap attempts, and fall back safely.
Imagine a vending machine that always drops something shaped like a snack — but it might be the wrong flavor, or an empty wrapper. The machine guarantees 'snack-shaped,' not 'the snack you ordered.' So you always check what fell out before handing it to a customer. If it's slightly wrong, you fix the small stuff yourself; if it's badly wrong, you press the button again and tell the machine exactly what was missing. You only retry a couple of times, and if it still fails, you give the customer a safe default instead of garbage. That checking step is the validate and repair loop.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Structured output is where a lot of LLM features quietly meet a hard systems requirement: a downstream service expects JSON of a precise shape, every single time, or it breaks. The naive assumption is that turning on the provider's JSON mode solves this. It helps, but it solves the wrong half of the problem, and teams that lean on it ship subtle data-corruption bugs.
The reason is a guarantee mismatch. JSON mode and constrained decoding promise syntactic validity — the bytes will parse as JSON. They say nothing about your schema: whether the required fields are present, the types are right, the enum is one of the allowed values, the number is in range. The model is still guessing the content, and a syntactically perfect object can be semantically wrong in ways your billing or fulfillment system can't tolerate.
This deep dive builds the validate and repair loop as the standard answer to that mismatch. We'll separate the two guarantees, walk the four stages of the loop, look hard at the repair path where the real design judgment lives, and cover the two things people forget — the attempt cap and the terminal fallback — plus the observability that keeps the whole thing honest in production.
Two guarantees, and why JSON mode only gives you one
The single most useful sentence in this whole topic is: 'is valid JSON' and 'matches my schema' are different guarantees. Everything in the design follows from keeping them separate.
JSON mode constrains the model's decoding so the output is well-formed JSON — balanced braces, quoted keys, no dangling syntax. That's genuinely useful; it removes a whole class of parse errors. But it operates at the grammar level. It has no idea that your schema requires an amount_cents integer field, that status must be one of four enum values, or that line_items can't be empty. The model fills those in by generation, and generation is a guess.
So a syntactically perfect object can still be wrong in exactly the ways that hurt: a missing required field, a string where you need a number, an out of range value, an invented enum. None of those are syntax errors; JSON mode waves them straight through.
That's why the validation gate is non-negotiable and why it must check your schema, not just 'did it parse.' Tools like Pydantic and Zod exist precisely to express that schema and enforce it. The model proposes; your validator disposes. Skip the gate and you've outsourced your data contract to a stochastic process — which is the original sin this whole pattern exists to prevent.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
def get_valid_json(prompt, schema, max_repairs=2):
messages = [{"role": "user", "content": prompt}]
for attempt in range(max_repairs + 1):
raw = model.create(messages, response_format="json_object")
try:
obj = repair_parse(raw) # fix fences/commas locally
return schema.validate(obj) # YOUR schema, not the provider's
except ValidationError as e:
log.warning("json_repair", attempt=attempt, error=str(e))
messages += [
{"role": "assistant", "content": raw},
{"role": "user", "content": f"That failed validation: {e}. Fix only that."},
]
return SAFE_DEFAULT # terminal fallback after the cap| Failure type | Repair route | Why |
|---|---|---|
| Trailing comma, code fence | Local repair-parse | Deterministic and free; no model round-trip |
| Missing required field | Re-ask with the error | Model must regenerate the value it omitted |
| Wrong type / out of range enum | Re-ask with the error | Targeted feedback converges faster than resampling |
| Failure persists past the cap | Safe default or controlled error | Never forward malformed data downstream |
Real products, models, and research that use this idea.
- OpenAI structured outputs and function calling constrain shape but production code still validates against a Pydantic or Zod schema before use.
- Instructor (Python) wraps the model call in Pydantic validation and automatically re-asks with the validation error on failure.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhen should you reach for constrained decoding or a strict tool schema instead of post-hoc validate and repair?
Push the guarantee into decoding when the provider supports schema-constrained generation; it lowers the failure rate at the source, though you still validate cross-field invariants the grammar can't express.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Trusting JSON mode to guarantee your schema, when it only guarantees syntactic JSON — required fields, types, and value ranges can still be wrong.
60 second bullets to scan on the way to the call.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.