Request: a model name plus a list of role-tagged messages (system / user / assistant), and sampling knobs like temperature, top_p, max_tokens, stop. Response: one or more assistant messages, a finish_reason (stop / length / tool_calls), and a usage block (prompt_tokens, completion_tokens, total_tokens). With stream=true the body becomes a sequence of SSE delta chunks instead of a single JSON payload.
The **chat completions** endpoint is the standard interface every modern hosted LLM exposes, originally shaped by OpenAI and now copied by every serious provider.
A request body has three required parts. First, a `model` field naming which model to use (for example `gpt-5.5` or `claude-sonnet-4-6`). Second, a `messages` array where each entry has a `role` (system, user, assistant, or tool) and `content` (the actual text). Third, optional sampling knobs like `temperature`, `top_p`, `max_tokens`, and `stop` sequences that control how the model decodes.
The response body comes back with one or more assistant messages in a `choices` array, plus two pieces of metadata that always matter. A `finish_reason` tells the caller why generation stopped: `stop` (natural end), `length` (hit max_tokens), or `tool_calls` (model wants to call a function). A `usage` block reports `prompt_tokens`, `completion_tokens`, and `total_tokens` for billing.
With `stream=true`, the response stops being one JSON object and becomes a sequence of Server-Sent Events. Each event carries an incremental delta until a final chunk with `finish_reason` arrives. GPT-5.5, Claude Sonnet 4.6, Gemini 3.1 Pro, and every open-source serving stack like vLLM all speak this shape.
An OpenAI-compatible **chat completions** request carries a model identifier, a role-tagged message list, and sampling parameters. A response carries the assistant's reply, a termination reason, and a token-usage block.
**Request fields that matter.** `model` selects the model. `messages` is an array of `{role, content}` pairs where role is one of system / user / assistant / tool, and content is text (or a multimodal array for vision endpoints). Sampling knobs include `temperature`, `top_p`, `max_tokens`, `stop` sequences, `presence_penalty`, `frequency_penalty`, and optional `tools` / `tool_choice` for function calling.
**Response fields that matter.** `choices[]` carries one or more `{message: {role, content}, finish_reason}` entries. `finish_reason` is the dispatch contract: `stop` means natural EOS, `length` means hit max_tokens, `tool_calls` means the model emitted a function call instead of text, `content_filter` means moderation tripped. The `usage` block reports prompt / completion / total tokens for billing.
**Streaming.** With `stream=true`, the server emits Server-Sent Events (SSE). Each event contains a `delta` (incremental content) and the final event carries the `finish_reason`. Anthropic adds a `message_start` event marking exact TTFT; OpenAI signals it with the first `delta` chunk. This shape is the de facto industry standard: Anthropic's Messages API, Gemini's generateContent, vLLM, TensorRT-LLM, and SGLang all expose OpenAI-compatible front doors for client compatibility.
Quick reference
TL;DR
Request carries model + role-tagged messages + sampling knobs; response carries assistant content + finish_reason + a usage block; streaming swaps the JSON body for SSE delta chunks.
Picture sending a structured letter to an assistant. The envelope says which assistant to deliver it to (the model name). Inside the envelope, every paragraph is labeled with who is speaking: system rules first, then a back and forth between user and assistant. You also include a sticky note with knobs like 'do not exceed 500 words' and 'be a little creative'. The assistant writes a reply with three things: the actual text, a stamp saying why she stopped writing, and a meter showing how many words you used in total. If you ticked the streaming box, instead of one finished letter you get a stack of postcards arriving one piece at a time.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Embedding workloads where there is no autoregressive generation Embedding endpoints take input text and return vectors; the chat completions message schema and finish_reason concept do not apply.
Pure batch inference with no real-time client Async batch APIs use file-based job submission with different response shapes optimized for cost rather than the chat completions per-request HTTP pattern.
Realtime audio or video streaming endpoints WebRTC and bidirectional streaming use a different protocol with frame-level chunks rather than SSE text deltas; the chat completions schema does not cover audio frames.
Real products, models, and research that use this idea.
1OpenAI's GPT-5.5 chat completions endpoint defines the canonical schema; every other provider adopts it or supplies an adapter.
2Anthropic's Messages API uses an OpenAI-compatible structure with role-tagged messages and adds explicit cache_creation / cache_read fields in the usage block.
3Gemini 3.1 Pro on Google AI Studio supports an OpenAI-compatible endpoint alongside its native generateContent API for ecosystem portability.
4vLLM, TensorRT-LLM, and SGLang all ship OpenAI-compatible HTTP servers so client SDKs work without modification against self-hosted Llama 4 Maverick or DeepSeek V4.
5OpenRouter and Together AI normalize dozens of model providers behind a single OpenAI-compatible chat completions URL.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does the chat completions contract handle multimodal inputs in 2026?
Sign in or go Pro to see the approach for this follow-up.
QWhy do self-hosted serving stacks bother implementing the OpenAI shape rather than their own API?
Sign in or go Pro to see the approach for this follow-up.
QWhat goes wrong if a server returns finish_reason=stop when the model actually wanted to call a tool?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Conflating the legacy completions endpoint with chat completions. Chat completions is structured around role-tagged messages, not a single prompt string, and finish_reason matters for tool-call dispatch.