Describe why structured output mode quietly raises the per call token bill

**Three independent forces push the realized cost up.** None of them is about the actual information the model returns. Two add tokens; one adds wall-clock time. First, **constrained decoding**. JSON mode forces the next-token distribution through a grammar or finite-state mask (Outlines, JSONSchema-FSM, llguidance). At every step the mask zeros out tokens that would violate the schema. That mask interacts badly with speculative decoding: candidate tokens from the draft model that violate the schema get rejected by the mask, so the verifier accepts fewer tokens per round. The model still runs at the same per-step bandwidth, but the per-token wall-clock effectively rises because speculation is less efficient. Second, **envelope tokens**. The JSON syntax itself is real billed output. Keys are repeated verbatim in every record (`"customer_name": ..., "order_id": ...`). Quotes, braces, brackets, commas, and escape sequences all count as output tokens. A free-text answer that would have been 80 tokens easily becomes 130 tokens when wrapped in JSON, a 60% uplift just from envelope. Third, **parse-failure retries**. The model occasionally emits a near-miss: an unclosed brace, an invalid escape, a value that fails schema validation. The application retries, paying for prompt input and full output a second time. Even a 2-3% failure rate adds up across a high-volume route. Combined, these typically land at a 10-40% net output-token bill increase over the free-text equivalent, depending on response shape. The fix is not to abandon structure; it is to use lean schemas, prefer providers with grammar-aware speculation, and budget for retries.

**Three compounding mechanisms.** None of them is about the underlying information; all of them add to the realized cost. First, **constrained decoding lowers speculation acceptance**. JSON mode runs every sampling step through a schema-aware grammar mask. The mask zeros out tokens that would violate the schema, so the next-token distribution narrows. Speculative decoding, which depends on the draft model producing candidates that match the target's distribution, takes a hit: candidates that the draft predicted but the mask forbids get rejected by the verifier. Acceptance rate drops, draft rounds yield fewer accepted tokens, and wall-clock latency per token rises even though output length is unchanged. Second, **the JSON envelope is real billed output**. Keys are repeated verbatim every record (`customer_name`, `order_id`, `address`). Quotes, braces, brackets, commas, and escape sequences are tokens. A response that would be 80 prose tokens easily becomes 130 JSON tokens. The uplift is biggest on short structured replies where the envelope dominates the payload, and smallest on long structured replies where the value content amortizes the envelope cost. Third, **parse failures trigger retries**. Even a well-tuned structured-output mode emits the occasional malformed payload: an unclosed string, an invalid escape, a value that violates schema validation. The application retries, paying for the input prompt and the output a second time. At a 2-3% failure rate, that adds 2-3% to the realized cost on top of the other axes. The combined effect lands at 10-40% above the free-text equivalent in practice. The fix is to use the leanest schema that works (shorter keys, fewer redundant fields), prefer providers with grammar-aware speculative decoding (some stacks adapt the draft model to respect the schema), cache the compiled grammar so the FSM is reused across requests, and budget explicitly for a small retry tail.