A 40-team org already has OpenAPI specs and three LLM providers. Defend layering MCP on top instead of picking one integration approach.

Short answer·Hard·4.0 · 0·~3 min·Relevant atAnthropic OpenAI

Attempt it

Your platform serves 40 internal teams shipping LLM features against OpenAI, Anthropic, and Gemini. You already have OpenAPI specs for every internal service. A staff engineer proposes adding MCP as a third integration layer on top of function calling and OpenAPI. Defend the three-layer architecture against the counter-argument that one standard should suffice.

Free · 2 AI evals / day

**MCP, function calling, and OpenAPI** are not competing standards you pick from. They are three layers, each sitting at a different boundary in the system. OpenAPI is the service contract. It defines what each service does, what endpoints it has, what data it accepts, and what it returns. At a 40-team organization your OpenAPI specs power frontend SDK generation, partner integrations, cron jobs, and every internal service call. None of those consumers have anything to do with LLMs. Replacing OpenAPI with MCP would break all of them. Vendor function calling is the format the LLM understands. Each provider (OpenAI, Anthropic, Gemini) requires tools in its own format. The host application is responsible for translating. You cannot skip this layer because the LLM API simply rejects anything else. MCP is the bridge between the host application and the tools it calls. You wrap each OpenAPI service in an MCP server (tools like openapi-mcp and FastAPI-MCP automate this). Every host that speaks MCP can then use any server without custom code. The math at 40 teams with 3 providers and multiple hosts: without MCP you build N hosts times M services in adapters. With MCP you build M wrappers once and each host connects through one protocol, shrinking the count from N times M down to N plus M. MCP wins when hosts need to mix tools across providers, when users plug servers into IDEs at runtime, or when the organization might swap providers. MCP loses on latency-critical paths (the JSON-RPC hop adds milliseconds), for single-purpose agents with a fixed tool set (the indirection adds nothing), and on prompt cache hot paths where a changing tool registry breaks the cache prefix. The 'just pick one' argument fails because removing any layer leaves a gap. Without OpenAPI, non-LLM consumers lose their contract. Without function calling, the LLM API rejects your request. Without MCP, every team rebuilds the adapter matrix from scratch.

**Three layers at three boundaries, not three alternatives.** OpenAPI is the service contract. At 40 teams with 120 services, the specs drive SDK generation, validation, partner docs, and audit. Auth is mature (Bearer, OAuth, mTLS). Non-LLM consumers (frontends, partners, cron jobs) depend on it. MCP cannot replace it because those consumers do not speak MCP. Vendor function calling is the LLM-to-host wire format. OpenAI, Anthropic, and Gemini each accept only their own tool format. The host translates. This layer is fixed by the provider and cannot be bypassed. **MCP is the host to tool boundary.** Wrap OpenAPI services in MCP servers (openapi-mcp, FastAPI-MCP auto-generate from specs). At 40 teams, 3 providers, N hosts: without MCP, N times M adapters per provider. With MCP, each service publishes one server, every host connects through one protocol, and the matrix collapses to N plus M. Tradeoff axes: cardinality (OpenAPI one-API-many-consumers, function calling per-provider, MCP one server many hosts), LLM-awareness (MCP concise, OpenAPI verbose), discovery (MCP has tools/list, OpenAPI static specs, function calling nothing), versioning (OpenAPI mature, MCP younger), auth (OpenAPI HTTP-level, MCP OAuth 2.1 for remote). MCP wins for multi-provider agentic apps, IDE/desktop plugins, and provider-swap environments. MCP loses for latency-critical paths (5 to 20 ms hop overhead), fixed single-purpose agents, and prompt cache hot paths where a churning registry invalidates the prefix. Defeating 'just pick one': OpenAPI answers 'what does the service do,' function calling answers 'how does the LLM ask,' MCP answers 'how does the host execute.' Each sits at a different boundary between different consumer sets.

Quick reference

TL;DR

OpenAPI, function calling, and MCP sit at three different boundaries (service contract, LLM wire format, host to tool execution) and removing any one breaks a different axis of the platform.

Memory aid

Easy to grasp

Picture a large company with restaurants, translators, and delivery drivers. Restaurants have menus that list everything they cook (that is OpenAPI describing what each service does). Customers speak different languages (those are the LLM providers, each with its own way of placing an order). Delivery drivers carry the food from the kitchen to the table (those are MCP servers connecting hosts to tools). If you remove the menus, nobody knows what the restaurant offers. If you remove the translators, customers cannot order in their language. If you remove the delivery drivers, every customer has to walk into every kitchen themselves, which gets chaotic at 40 restaurants and three languages. Each role handles a different problem, and saying 'just pick one' is like insisting restaurants only need menus but not drivers.

Key concepts

Function Calling OpenAPI LLM Tool Use Platform Architecture

Detailed answer & concept explanation

~9 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

Open by framing MCP, function calling, and OpenAPI as three layers at three boundaries, not three alternatives. Walk each layer: OpenAPI as the service contract (consumed by non-LLM clients, mature auth, versioning), function calling as the LLM-to-host wire format (fixed by the provider, cannot be bypassed), MCP as the host to tool boundary (wraps OpenAPI, runtime tools/list, LLM-aware descriptions). Quantify the N times M to N plus M math at 40 teams, 3 providers, and multiple hosts. Cover tradeoff axes (cardinality, LLM-awareness, discovery, versioning, auth). Name when MCP wins and loses. Close by defeating the 'just pick one' counter.

Real-world examples

Real products, models, and research that use this idea.

openapi-mcp and FastAPI-MCP are auto-generators that produce an MCP server from an OpenAPI spec, letting platform teams wrap services at scale without hand-writing each server.
Claude Desktop, Cursor, Zed, and Claude Code all consume MCP servers through the same protocol while translating to Anthropic's or OpenAI's function-calling format internally, validating the three-layer separation in production.
GitHub publishes a first-party MCP server (rewritten in Go in 2025) alongside its REST and GraphQL APIs, keeping the OpenAPI contract for non-LLM clients and adding MCP as a derived view for AI hosts.
The 2023 GPT plugin manifest tried to collapse OpenAPI and tool use into a single layer and did not see wide adoption; the ecosystem converged on MCP-wraps-OpenAPI as the cleaner separation by 2026.
Stripe, Sentry, Slack, and similar SaaS vendors ship MCP servers as thin wrappers that add LLM-aware descriptions to the underlying API operations, while their REST APIs remain the canonical contracts.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QTrace a single request end to end through the three-layer model, from user query to backend response. Which component does what at each boundary?

QA team argues: 'let agents call OpenAPI services directly via HTTP tool calls and skip MCP.' What do they lose?

QHow exactly does adopting MCP put prompt caching at risk, and what mitigations exist?

QMCP versioning is less mature than OpenAPI's. What does a pragmatic per-server versioning discipline look like in 2026?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating MCP, function calling, and OpenAPI as interchangeable standards where choosing one eliminates the need for the other two. Each layer answers a distinct question no other layer can.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.