When does grammar-constrained decoding (Outlines, llama.cpp grammars) actually beat a post hoc JSON validate and repair loop?

**Grammar-constrained decoding** restricts the model's token choice at each step to tokens that keep the output syntactically valid. The correct answer is B; it wins in three specific regimes. The contrast is between two approaches to structured output. The grammar approach (Outlines, llama.cpp grammars, some local inference servers) builds a state machine over the schema and masks any token at each decode step that would break it. The output is guaranteed valid. The cost is roughly 10 to 20% extra latency and a slight quality hit because the constraint can rule out reasoning paths the model would have taken. The repair approach is simpler: let the model generate freely, parse the result, and if it fails, ask the model to fix it. About 95% of calls succeed first try; the other 5% pay a retry round trip. **When grammar wins.** Zero tolerance environments (a downstream parser will crash, an embedded system has no error path, a regulator requires guaranteed schema compliance). High QPS systems where a 5% retry rate times millions of QPS adds up to more compute than the 15% per call grammar overhead. Regulated industries where 'we sometimes return malformed output' is not an acceptable answer. **When repair wins.** Most other cases. Lower latency in the common path, full access to the model's reasoning, easy to mix with hosted APIs that do not expose token level grammars. The other options are tempting traps. Option A overclaims; grammar is not strictly better. Option C is wrong; hosted Structured Outputs from OpenAI and Anthropic do offer schema enforcement but it is a different mechanism. Option D invents a depth threshold that does not exist.

**Grammar-constrained decoding** beats a repair loop in three named regimes: zero tolerance for malformed output, repair cost dominating at scale, and regulated environments where format compliance is mandatory. Answer B. Grammar based decoders (Outlines, llama.cpp grammars, XGrammar) restrict the next-token distribution to tokens that keep the output valid against a schema or grammar. Output is guaranteed parseable. The cost is roughly 10 to 20% latency and a measurable quality hit on harder tasks because the constraint can prune reasoning paths. Repair loops generate freely, validate, and retry on failure. They are cheaper in the common case because there is no decode time overhead and no reasoning restriction. The long tail can be expensive though. 5% retries at high QPS dominate the grammar latency cost. Any unrecoverable failure either crashes the downstream parser or requires fallback logic. The three winning regimes for grammar are concrete. Embedded systems or downstream parsers that cannot recover from invalid input. High QPS production where retry economics flip. Regulated environments (finance, healthcare, gov) where 'sometimes invalid' is unacceptable. Option A overstates the win; grammar is not strictly better. Option C confuses grammar based decoding with hosted Structured Outputs, which are related but different. Option D invents a depth threshold; repair loops handle deep nesting fine.