Spot the flaw in a JSON mode handler that trusts the model's output
Click any words you think contain an error. Click again to unmark.
The flaw is the closing 'because' clause: JSON mode guarantees syntax, not your schema. The handler forwards unvalidated output to billing, so a missing field or wrong type silently corrupts data.
Imagine a delivery service that promises every package will arrive sealed and unbroken. That's a real promise — but it says nothing about what's inside. You could get a sealed box with the wrong item. This code makes the same mistake. JSON mode promises the model's answer is a properly sealed JSON box. The code then assumes the contents must also be correct and ships the box straight to billing. The fix is to open the box and check the contents against your packing list before you forward it. That check is schema validation, and it's exactly the step the handler skips.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Spot-the-error questions reward one skill above all: locating the exact sentence where the reasoning goes wrong, rather than fixing whatever looks unfamiliar. In this snippet the parsing is fine, the forwarding is mechanically fine, and the tempting wrong answer is to flag JSON.parse for lacking a try/catch. That's a real but minor issue. The load-bearing defect is the justification at the end.
The handler asserts that JSON mode 'guarantees the output is always valid and correctly shaped,' and uses that to skip validation before sending data to billing. The entire bug is compressed into the word 'because.' If that clause were true, the design would be defensible. It isn't true, and recognizing precisely why is the point of the exercise.
This deep dive walks through how to find the flaw fast, the syntax versus schema distinction that the snippet conflates, the concrete failure cases that slip through a parse-only handler, why the billing sink turns a correctness bug into a money bug, and the validation-gate fix with its repair and fallback path. The goal is to make this category of mistake instantly recognizable in any structured-output code you review.
How to find the flaw fast: follow the 'because'
A reliable technique for spot the error prompts is to hunt for the justifying clause — the part that says why something is safe to do — because that's where unstated assumptions hide. Here the suspect clause is unmissable once you look for it: 'because JSON mode guarantees the output is always valid and correctly shaped.'
Every design decision in the snippet hangs off that clause. The decision to skip schema validation, the decision to forward straight to billing — both are only acceptable if the clause holds. So you don't even need to scrutinize the parsing logic first. You test the justification, and if it fails, every choice it licensed is suspect.
The trap answer is to flag JSON.parse for being unguarded. It's a legitimate secondary issue — an unexpected edge case can make it throw — but it's not why this code corrupts billing data. Fixing only the parse (wrapping it in try/catch) would still forward schema-invalid objects happily. A senior reviewer names the primary flaw first and mentions the parse guard as a footnote.
The meta-lesson: in structured-output code, the dangerous bugs are usually assumptions about what a 'guarantee' covers, not visible logic errors. Train your eye on the claims, not just the control flow.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
// B---BUG: trusts JSON mode, forwards unvalidated
// const obj = JSON.parse(raw); billing.charge(obj);
// FIX: guard the parse, validate against YOUR schema first
let obj;
try {
obj = OrderSchema.parse(JSON.parse(raw)); // throws on bad syntax OR bad schema
} catch (e) {
obj = await repairOrReask(raw, e); // capped repair/re-ask path
if (!obj) return safeDefault(); // terminal fallback
}
billing.charge(obj);| Claim in the code | What JSON mode actually gives | Gap |
|---|---|---|
| 'always valid' | Parses as JSON syntax | Parse can still throw on edge cases |
| 'correctly shaped' | Nothing about your fields/types | Missing/extra fields, wrong types |
| safe to forward to billing | No semantic guarantee at all | Out of range or invented values flow through |
Real products, models, and research that use this idea.
- OpenAI JSON mode and structured outputs constrain shape, but production handlers still validate with Pydantic or Zod before any side effect.
- Stripe-style billing integrations reject malformed amounts at an explicit schema boundary rather than trusting an upstream payload.
What an interviewer would ask next. Try answering before peeking at the approach.
QStrict schema-constrained decoding is available. Does that let you safely drop the validation gate?
It lowers failure rate at the source but can't enforce cross-field invariants or business ranges; keep a thin validation gate even with constrained decoding.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Reading 'JSON mode' as a guarantee of your schema. It only guarantees the bytes parse as JSON — fields, types, and ranges are still the model's guess.
60 second bullets to scan on the way to the call.
Why the 'because' clause is the actual flaw, not the parse call
The distinction between syntactic JSON and your schema
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.