Give the shape of an OpenAI compatible chat completions request and response.

Flashcard·Easy·4.0 · 0·~30s·Asked atAi4bharat Citadel Krutrim·Relevant atOpenAI

Attempt it

Request: a model name plus a list of role-tagged messages (system / user / assistant), and sampling knobs like temperature, top_p, max_completion_tokens, stop. Response: one or more assistant messages, a finish_reason (stop / length / tool_calls), and a usage block (prompt_tokens, completion_tokens, total_tokens). With stream=true the body becomes a sequence of SSE delta chunks instead of a single JSON payload.

Your answer

The **chat completions** endpoint is the standard interface that hosted LLMs expose. OpenAI first defined it, and every major provider now copies it. A request body has three parts. First, a `model` field naming which model to use (for example `gpt-5.5` or `claude-sonnet-4-6`). Second, a `messages` array where each entry has a `role` (developer or system, user, assistant, or tool) and `content` (the actual text). Third, optional sampling knobs like `temperature`, `top_p`, `max_completion_tokens`, and `stop` sequences that control how the model decodes. The response body comes back with one or more assistant messages in a `choices` array, plus two pieces of metadata. A `finish_reason` tells the caller why generation stopped: `stop` (natural end), `length` (hit the token limit), or `tool_calls` (model requests a function call). A `usage` block reports `prompt_tokens`, `completion_tokens`, and `total_tokens` for billing. With `stream=true`, the response stops being one JSON object and becomes a sequence of Server-Sent Events. Each event carries an incremental delta until a final chunk with `finish_reason` arrives. GPT-5.5, Claude Sonnet 4.6, Gemini 3.1 Pro and every open-source serving stack such as vLLM all speak this shape.

An OpenAI-compatible **chat completions** request carries a model identifier, a role-tagged message list, and sampling parameters. A response carries the assistant's reply, a termination reason, and a token-usage block. **Request fields that matter.** `model` selects the model. `messages` is an array of `{role, content}` pairs where role is one of developer / system / user / assistant / tool, and content is text (or a multimodal array for vision endpoints). Sampling knobs include `temperature`, `top_p`, `max_completion_tokens`, `stop` sequences, `presence_penalty` and `frequency_penalty`. Optional `tools` and `tool_choice` enable function calling. **Response fields that matter.** `choices[]` carries one or more `{message: {role, content}, finish_reason}` entries. `finish_reason` is the dispatch contract: `stop` means natural EOS, `length` means the token limit was hit, `tool_calls` means the model emitted a function call instead of text, `content_filter` means moderation tripped. The `usage` block reports prompt / completion / total tokens for billing. **Streaming.** With `stream=true`, the server emits Server-Sent Events (SSE). Each event contains a `delta` (incremental content) and the last data chunk carries the `finish_reason`. TTFT is the arrival of the first chunk that carries content. This shape is the de facto industry standard. Anthropic and Google offer OpenAI-compatible endpoints next to their native Messages and generateContent APIs, and vLLM, TensorRT-LLM and SGLang ship OpenAI-compatible servers.

TL;DR

Request carries model + role-tagged messages + sampling knobs; response carries assistant content + finish_reason + a usage block; streaming swaps the JSON body for SSE delta chunks.

Memory aid

Easy to grasp

Picture sending a structured letter to an assistant. The envelope says which assistant to deliver it to (the model name). Inside the envelope, every paragraph is labeled with who is speaking: system rules first, then a back and forth between user and assistant. You also include a sticky note with knobs like 'do not exceed 500 words' and 'be a little creative'. The assistant writes a reply with three things: the actual text, a stamp saying why she stopped writing, and a meter showing how many words you used in total. If you ticked the streaming box, instead of one finished letter you get a stack of postcards arriving one piece at a time.

Key concepts

Chat completions endpoint OpenAI-compatible API Streaming inference Sampling parameters Tokenizer

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

The chat completions endpoint is the common request format of hosted LLM serving. OpenAI's shape, introduced in 2023, became the de facto industry standard because every client SDK and agent framework was written against it. Every major provider (Anthropic, Google, Mistral, DeepSeek) ships either an OpenAI-compatible endpoint or a thin adapter. Every open-source serving stack (vLLM, TensorRT-LLM, SGLang) ships one too, so self-hosted deployments work with existing client code after a base-URL change.

The request schema in detail

A chat completions request body has three components.

model, a string identifying which model to use. Hosted APIs accept model IDs such as gpt-5.5 and claude-sonnet-4-6. Self-hosted vLLM accepts the model path or alias configured at startup.
messages, an array of message objects, each with a role and content. The roles are developer or system (instructions and persona), user (input from the caller), assistant (prior model output, used for multi-turn context), and tool (results from function calls being fed back). Content is text by default; for multimodal endpoints it is an array of typed parts (text, image, audio).
Sampling and shaping parameters, temperature (rescales logits), top_p (nucleus truncation), max_completion_tokens (hard cap on generated tokens, including reasoning tokens), stop (sequences that terminate generation), presence_penalty and frequency_penalty (discourage repetition), seed (best-effort reproducibility), and tools / tool_choice for function calling.

Multimodal content arrays

When content is an array, each entry has a type field (text, image_url, input_audio). GPT-5.5 and Claude Opus 4.7 both accept this shape. Image parts can carry a base64 data URL or a remote URL. Vision models turn each image into vision tokens; the count depends on the image size and the provider, and those tokens are billed as input in the usage block.

The response schema and finish_reason as a contract

Streaming: SSE framing and TTFT

Why every stack speaks the OpenAI shape

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

OpenAI's Chat Completions API defines the reference schema; every other provider adopts it or offers an adapter.
Anthropic's native Messages API uses a similar structure with role-tagged messages, and its usage block adds cache_creation_input_tokens and cache_read_input_tokens fields. Anthropic also offers an OpenAI SDK compatibility endpoint, mainly for testing and comparing models.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does the chat completions contract handle multimodal inputs?

The content field per message becomes an array of typed parts: text parts, image parts (with base64 or URL), and audio parts on audio-capable models. GPT-5.5, Claude Opus 4.7 and Gemini 3.1 Pro all use this shape. Image tokenization is provider-specific, and image tokens count toward input usage.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Conflating the legacy completions endpoint with chat completions. Chat completions is structured around role-tagged messages, not a single prompt string, and finish_reason matters for tool-call dispatch.

60 second bullets to scan on the way to the call.

Required request fields: model, messages, optional sampling knobs
The message roles (developer or system, user, assistant, tool)

Primary sources. Browse if you want the original framing.