Structured Output
Also known as: JSON mode, Constrained decoding
Force the model's output to match a schema: JSON, regex, or a context-free grammar.
Constraining an LLM to produce output that conforms to a specific schema (JSON, regex, grammar). Implemented via prompt-and-pray, retry loops, or constrained decoding that masks invalid tokens at sample time.
In practice
Foundational for any LLM-as-API integration. Expect questions on outlines/llguidance, OpenAI JSON mode, and failure modes.
How it compares
Function calling is a specific case of structured output (schema = tool signature); structured output is the general idea.
Related topics
Questions that mention this term
- Flashcard: what is a stop sequence in an LLM API call and what is it used for?
- Is trimming the prompt usually the biggest cost lever on a chat endpoint?
- Which structural property…
- Design the structural layout for a production prompt powering a medical document Q&A system over retrieved clinician notes.
- Why cap tool output size at the tool boundary instead of letting the model deal with it?
- Walk through the LangGraph supervisor pattern for multi-agent orchestration
Related terms
Agent Loop
The runtime: LLM call → tool call → observation → LLM call → … until a final answer or step cap.
AI Agents
LLMs that loop: plan → call tools → observe results → repeat until done.
Chain-of-Thought (CoT)
Ask the model to think step by step before answering. It boosts accuracy on reasoning tasks.
Context Rot
Long-running chats degrade: early instructions get forgotten, and tool calls become less reliable.
Function Calling
The model emits structured JSON specifying a tool call instead of free text; the host executes it.
In-Context Learning (ICL)
Show the model a few examples in the prompt and it learns the pattern, no fine-tuning needed.