Server-Sent Events streaming: define it and explain why chat APIs default to it

Flashcard·Easy·4.0 · 0·~30s·Asked atMeesho Uniphore Unity·Relevant atOpenAI

Attempt it

SSE is a one-way HTTP streaming protocol where the server pushes a sequence of small data: events to a client over a single long-lived response, consumed in the browser by the EventSource API. Chat APIs (OpenAI, Anthropic, vLLM) default to it because each generated token can be flushed as it arrives, so the user sees output at TTFT instead of waiting for the full completion. It rides on plain HTTP, traverses proxies cleanly, and matches the one-way shape of token output without the weight of WebSockets.

Your answer

**Server-Sent Events (SSE)** is a way for a web server to push a stream of small messages to a browser over a single long-lived HTTP response. The server opens the response, sets the content type to `text/event-stream`, and then writes one `data:` line at a time as new information becomes available. The browser reads each line as it arrives. LLM chat APIs use SSE because their output is a stream of tokens that get generated one after another. Without streaming, the client would sit silent until the entire reply was ready, which feels broken for anything longer than a few words. With SSE, the server flushes each new token immediately, so the user starts reading at TTFT instead of waiting for the whole response. SSE has three properties that make it the natural fit. It rides on plain HTTP, so any proxy, load balancer, or CDN that handles HTTP handles it. It is one-way: server to client only, which matches the shape of token output. And it is much lighter than WebSockets, which require a protocol upgrade and a two-way channel the chat case does not need. OpenAI, Anthropic, vLLM, and TGI all expose SSE as the default streaming format.

**SSE (Server-Sent Events)** is a unidirectional HTTP streaming protocol where the server holds a single response open and pushes a sequence of `data:` events to the client as they become available. The response carries `Content-Type: text/event-stream`, and the client consumes it via the browser's `EventSource` API or any HTTP library that supports chunked reads. For an LLM API, the value is direct. Token generation is sequential and slow on a per-token basis, but the first token is usable on its own. Streaming via SSE lets the server flush each generated token as soon as the decode step completes, so the user-visible latency collapses from end to end response time to TTFT. The rest of the tokens arrive at the steady-state decode rate (`1 / TPOT`), but the experience already feels responsive. ```text HTTP/1.1 200 OK Content-Type: text/event-stream data: {"choices":[{"delta":{"content":"The"}}]} data: {"choices":[{"delta":{"content":" answer"}}]} data: [DONE] ``` SSE wins over alternatives for this exact shape. Polling burns RTTs and misses tokens. WebSockets are bidirectional and require an HTTP upgrade handshake plus full-duplex framing the chat case does not need. SSE rides on plain HTTP/1.1 or HTTP/2, traverses corporate proxies and CDNs without special handling, and reconnects via `Last-Event-ID` semantics. OpenAI's Chat Completions, Anthropic Messages, vLLM, and Hugging Face TGI all expose SSE as the default streaming transport for these reasons.

TL;DR

SSE is a one-way HTTP streaming protocol; chat APIs use it so each generated token can be flushed to the client the moment it is produced.

Memory aid

Easy to grasp

Imagine watching a slow printer dispense a long receipt. Instead of waiting for the whole receipt and then handing it to you, the printer slides each line out as soon as the ink dries. SSE is that printer for an LLM. The server opens one ordinary HTTP connection to your browser and keeps it open. Every time the model generates a new token, the server pushes a tiny event down that pipe, and the browser stitches the events together on screen. Compared to alternatives, SSE is simple: plain HTTP, no upgrade handshake, no two-way channel needed because the server is the only one talking. Your code just opens the URL, listens, and renders text as it arrives.

Key concepts

Streaming TTFT Decoding LLM Serving

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Server-Sent Events is the boring, dependable plumbing under every modern chat completion stream. It is not a new invention; the WHATWG specified it in 2009 and browsers have shipped it since then. What changed in 2023 to 2026 is that every LLM provider made it the default transport for chat APIs, because the shape of token generation matches what SSE was built for almost exactly.

This deep dive covers what SSE actually is at the wire level, why it fits LLM streaming better than the alternatives, how the major providers frame their events, and which production gotchas separate a smooth streaming deployment from one that silently breaks user-visible latency.

What SSE is on the wire

SSE is a single long-lived HTTP response with Content-Type: text/event-stream. The body is a sequence of events; each event is a small text record terminated by a blank line.

The minimum frame is one data: line and a blank line:

text

data: {"token":"hello"}

data: {"token":" world"}

The spec also defines event: (event type), id: (last event id for resumption), and retry: (reconnect hint) fields, plus a comment syntax (: keepalive) used for heartbeat pings. The browser consumes the stream via new EventSource(url), which fires message events for each data: chunk. The protocol is plain HTTP/1.1 chunked transfer encoding (or HTTP/2 / HTTP/3 streams), no upgrade handshake, no separate framing layer.

The LLM case bends one convention: the request is usually a POST with a JSON body (the prompt), not a GET, because the EventSource API does not support custom headers or bodies. Production clients therefore use a fetch call with a streaming body reader rather than the native EventSource. The response shape is identical.

Why this shape fits LLM streaming

How major providers frame their events

Production gotchas

When SSE is the wrong choice

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

OpenAI Chat Completions with `stream: true` returns `text/event-stream` and pushes `data: {delta}` chunks plus a final `data: [DONE]` sentinel.
Anthropic Messages streaming sends typed SSE events like `message_start`, `content_block_delta`, and `message_stop` for structured client parsing.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does the server detect a client disconnect during an SSE stream, and why does that matter for cost?

Writes to a closed socket return EPIPE; the runtime treats that as a cancel signal and stops decoding the request, releasing its KV cache slot. Without this, abandoned requests keep consuming HBM and slowing every other in-flight request.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Thinking SSE is bidirectional like WebSockets. SSE is server to client only over plain HTTP; the client uses a separate POST for the request that opened the stream.

60 second bullets to scan on the way to the call.

Define SSE as a unidirectional HTTP streaming protocol.
Name the response content type and the browser API that consumes it.

Primary sources. Browse if you want the original framing.