**Function calling** is a provider-side feature in modern LLM APIs (OpenAI tools, Anthropic tool_use, Google function_declarations) where the model emits a structured tool-call JSON instead of plain text when it decides to invoke a function.
The flow is straightforward. You declare available functions to the API with their schemas. When the model decides to call one, the API returns a structured tool-call object: the function name plus arguments as JSON. The runtime parses that JSON and executes the corresponding function. The result is sent back as a tool-result message, and the model continues.
The key property is that the structured output is guaranteed at the token-sampling layer. The model is not writing JSON-like text that you then parse with regex; the API enforces well-formedness during sampling, so parsing is deterministic.
Why this matters for agent loops. Every multi-step pattern (ReAct, Plan-and-Execute, agentic RAG) has an Action step where the model picks a tool. With function calling, that step emits clean structured JSON every time, and the runtime can route to the right tool with confidence. Function calling is the substrate that makes tool-using agents practical at scale.
**Function calling** is a provider-side API feature where the LLM emits a structured tool-call JSON object instead of plain text when it decides to invoke a function. It is provided by OpenAI (as 'tools'), Anthropic (as 'tool_use' content blocks), Google (as 'function_declarations'), and most modern open-weight models (Llama 4, Mistral Large 3, Qwen 3.5, DeepSeek V4).
The flow has four steps. The developer declares available functions to the API with their JSON Schema. During inference, the model decides whether to emit text or a tool call. When it emits a tool call, the API returns a structured object with the function name and arguments. The runtime parses the object, executes the function, and sends the result back as a tool-result message.
The load-bearing property is that the structure is guaranteed at the token-sampling layer. Providers enforce well-formedness during sampling (via constrained decoding or grammar-aware sampling), so the JSON is always parseable.
Why this matters for agent loops. Every multi-step pattern (ReAct, Plan-and-Execute, agentic RAG) has an Action step where the model picks a tool. Before function calling, this step relied on parsing free text with regex, which was fragile. Function calling makes the Action step deterministic and reliable, which is what made tool-using agents practical at production scale.
Function calling is the inference-layer mechanism; MCP is the complementary system-layer protocol. Together they form the substrate for modern agents.
Quick reference
TL;DR
Function calling is a provider-side feature where the model emits structured tool-call JSON instead of plain text. The deterministic parseability is what makes tool-using agent loops practical at scale.
Imagine asking an assistant to look something up. One way: they write a sentence saying 'please search for X' and you read the sentence and try to figure out what they meant. The other way: they hand you a small form with the search field filled in, so there is no ambiguity about what they want. Function calling is the second way. Modern LLM APIs let the model produce a structured tool-call form (a JSON object with the function name and arguments) instead of a free-text request. The runtime can read the form directly and call the right function. No guessing, no regex. This deterministic shape is what makes agent loops reliable enough to ship in production.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Using a model that does not support native function calling. Older or smaller models without function-calling support force you back to text-parsed tool calls, which is fragile but sometimes the only option.
Trying to get rich tool-call reasoning out of free text on a function-calling model. If you describe tools in the system prompt and ask the model to emit them as text, you are bypassing the API guarantee and reintroducing parser fragility.
Assuming function calling solves semantic correctness as well as structural parseability. Function calling guarantees well-formed JSON, but the model can still pick the wrong tool or supply wrong arguments; runtime validation and error handling are still needed.
Quick scan · 2–4 min
Everything important, quickly.
Define function calling as the provider-side feature where the model emits structured tool-call JSON guaranteed at the sampling layer. Name the providers (OpenAI, Anthropic, Google, open-weight). Contrast with prompt-engineered tool calls. Explain why this is the substrate for reliable agent loops. Distinguish from MCP. Close with the known limit: structure is guaranteed, semantic correctness is not.
Real-world examples
Real products, models, and research that use this idea.
1OpenAI's tools parameter in chat completions returns tool_calls in the response; GPT-5.5 and o3 both support function calling natively.
2Anthropic's Claude Opus 4.7 and Sonnet 4.6 support tool_use content blocks, the structured tool-call format that LangGraph and CrewAI build their agent loops around.
3Google's Gemini 3.1 Pro uses function_declarations; the structured function_call response part is consumed by Vertex AI agent runtimes.
4Open-weight models like Llama 4 Maverick, Mistral Large 3, Qwen 3.5, and DeepSeek V4 all ship with tool-use support, often using OpenAI-compatible APIs so existing agent code works without changes.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow is function calling different from MCP?
Sign in or go Pro to see the approach for this follow-up.
QWhy is function calling more reliable than prompt-engineered tool calls?
Sign in or go Pro to see the approach for this follow-up.
QWhat does function calling not solve?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Confusing function calling with prompt-engineered tool calls. Prompt-engineered tool calls rely on the model writing parseable text; function calling guarantees structured JSON at the token-sampling layer.