Walk through the shape of an OpenAI compatible chat completions request and response.

Flashcard·Easy·4.0 · 0·~30s·Asked atAi4bharat Citadel Krutrim·Relevant atOpenAI

Attempt it

Request: a model name plus a list of role-tagged messages (system / user / assistant), and sampling knobs like temperature, top_p, max_tokens, stop. Response: one or more assistant messages, a finish_reason (stop / length / tool_calls), and a usage block (prompt_tokens, completion_tokens, total_tokens). With stream=true the body becomes a sequence of SSE delta chunks instead of a single JSON payload.

Your answer

The **chat completions** endpoint is the standard interface every modern hosted LLM exposes, originally shaped by OpenAI and now copied by every serious provider. A request body has three required parts. First, a `model` field naming which model to use (for example `gpt-5.5` or `claude-sonnet-4-6`). Second, a `messages` array where each entry has a `role` (system, user, assistant, or tool) and `content` (the actual text). Third, optional sampling knobs like `temperature`, `top_p`, `max_tokens`, and `stop` sequences that control how the model decodes. The response body comes back with one or more assistant messages in a `choices` array, plus two pieces of metadata that always matter. A `finish_reason` tells the caller why generation stopped: `stop` (natural end), `length` (hit max_tokens), or `tool_calls` (model wants to call a function). A `usage` block reports `prompt_tokens`, `completion_tokens`, and `total_tokens` for billing. With `stream=true`, the response stops being one JSON object and becomes a sequence of Server-Sent Events. Each event carries an incremental delta until a final chunk with `finish_reason` arrives. GPT-5.5, Claude Sonnet 4.6, Gemini 3.1 Pro, and every open-source serving stack like vLLM all speak this shape.

An OpenAI-compatible **chat completions** request carries a model identifier, a role-tagged message list, and sampling parameters. A response carries the assistant's reply, a termination reason, and a token-usage block. **Request fields that matter.** `model` selects the model. `messages` is an array of `{role, content}` pairs where role is one of system / user / assistant / tool, and content is text (or a multimodal array for vision endpoints). Sampling knobs include `temperature`, `top_p`, `max_tokens`, `stop` sequences, `presence_penalty`, `frequency_penalty`, and optional `tools` / `tool_choice` for function calling. **Response fields that matter.** `choices[]` carries one or more `{message: {role, content}, finish_reason}` entries. `finish_reason` is the dispatch contract: `stop` means natural EOS, `length` means hit max_tokens, `tool_calls` means the model emitted a function call instead of text, `content_filter` means moderation tripped. The `usage` block reports prompt / completion / total tokens for billing. **Streaming.** With `stream=true`, the server emits Server-Sent Events (SSE). Each event contains a `delta` (incremental content) and the final event carries the `finish_reason`. Anthropic adds a `message_start` event marking exact TTFT; OpenAI signals it with the first `delta` chunk. This shape is the de facto industry standard: Anthropic's Messages API, Gemini's generateContent, vLLM, TensorRT-LLM, and SGLang all expose OpenAI-compatible front doors for client compatibility.

TL;DR

Request carries model + role-tagged messages + sampling knobs; response carries assistant content + finish_reason + a usage block; streaming swaps the JSON body for SSE delta chunks.

Memory aid

Easy to grasp

Picture sending a structured letter to an assistant. The envelope says which assistant to deliver it to (the model name). Inside the envelope, every paragraph is labeled with who is speaking: system rules first, then a back and forth between user and assistant. You also include a sticky note with knobs like 'do not exceed 500 words' and 'be a little creative'. The assistant writes a reply with three things: the actual text, a stamp saying why she stopped writing, and a meter showing how many words you used in total. If you ticked the streaming box, instead of one finished letter you get a stack of postcards arriving one piece at a time.

Key concepts

Chat completions endpoint OpenAI-compatible API Streaming inference Sampling parameters Tokenizer

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

The chat completions endpoint is the lingua franca of hosted LLM serving in 2026. OpenAI's original shape, introduced in 2023, became the de facto industry standard because every client SDK and agent framework was written against it. Every serious provider (Anthropic, Google, Mistral, DeepSeek) now ships either a native OpenAI-compatible endpoint or a thin adapter, and every open-source serving stack (vLLM, TensorRT-LLM, SGLang) ships the front door so self-hosted deployments slot into existing client code with one base-URL change.

This deep dive walks through the request and response schemas in detail, the role of finish_reason as a dispatch contract, how streaming changes the wire format, and the production concerns that distinguish a working implementation from a correct one.

The request schema in detail

A chat completions request body has three components.

model, a string identifying which model to use. Hosted APIs accept names like gpt-5.5, claude-sonnet-4-6, gemini-3.1-pro. Self-hosted vLLM accepts the model path or alias configured at startup.
messages, an array of message objects, each with a role and content. The four roles are system (instructions and persona), user (input from the caller), assistant (prior model output, used for multi-turn context), and tool (results from function calls being fed back). Content is text by default; for multimodal endpoints it is an array of typed parts (text, image, audio).
Sampling and shaping parameters, temperature (rescales logits), top_p (nucleus truncation), max_tokens (hard cap on output length), stop (sequences that terminate generation), presence_penalty and frequency_penalty (discourage repetition), seed (best-effort reproducibility), and tools / tool_choice for function calling.

Multimodal content arrays

When content is an array, each entry has a type field (text, image_url, input_audio). GPT-5.5 and Claude Opus 4.7 both accept this shape. Image parts can carry a base64 data URL or a remote URL; vision models tokenize the image into a fixed number of vision tokens (often around 1024 for high-res images) which are billed in the usage block.

The response schema and finish_reason as a contract

Streaming: SSE framing and TTFT

Why every stack speaks OpenAI shape in 2026

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

OpenAI's GPT-5.5 chat completions endpoint defines the canonical schema; every other provider adopts it or supplies an adapter.
Anthropic's Messages API uses an OpenAI-compatible structure with role-tagged messages and adds explicit cache_creation / cache_read fields in the usage block.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does the chat completions contract handle multimodal inputs in 2026?

The content field per message becomes an array of typed parts: text parts, image parts (with base64 or URL), and increasingly audio parts. GPT-5.5, Claude Opus 4.7, and Gemini 3.1 Pro all use this shape. Tokenization for image parts is provider-specific and billed at separate rates.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Conflating the legacy completions endpoint with chat completions. Chat completions is structured around role-tagged messages, not a single prompt string, and finish_reason matters for tool-call dispatch.

60 second bullets to scan on the way to the call.

Required request fields: model, messages, optional sampling knobs
The four message roles (system, user, assistant, tool)

Primary sources. Browse if you want the original framing.