Server-Sent Events streaming: define it and explain why chat APIs default to it

Flashcard·Easy·4.0 · 0·~30s

Attempt it

SSE is a one-way HTTP streaming protocol where the server pushes a sequence of small data: events to a client over a single long-lived response, consumed in the browser by the EventSource API. Chat APIs (OpenAI, Anthropic, vLLM) default to it because each generated token can be flushed as it arrives, so the user sees output at TTFT instead of waiting for the full completion. It rides on plain HTTP, traverses proxies cleanly, and matches the one-way shape of token output without the weight of WebSockets.

Your answer

**Server-Sent Events (SSE)** is a way for a web server to push a stream of small messages to a browser over a single long-lived HTTP response. The server opens the response, sets the content type to `text/event-stream`, and then writes one `data:` line at a time as new information becomes available. The browser reads each line as it arrives. LLM chat APIs use SSE because their output is a stream of tokens that get generated one after another. Without streaming, the client would sit silent until the entire reply was ready, which feels broken for anything longer than a few words. With SSE, the server flushes each new token immediately, so the user starts reading at TTFT instead of waiting for the whole response. SSE has three properties that make it the natural fit. It rides on plain HTTP, so any proxy, load balancer, or CDN that handles HTTP handles it. It is one-way: server to client only, which matches the shape of token output. And it is much lighter than WebSockets, which require a protocol upgrade and a two-way channel the chat case does not need. OpenAI, Anthropic, vLLM, and TGI all expose SSE as the default streaming format.

**SSE (Server-Sent Events)** is a unidirectional HTTP streaming protocol where the server holds a single response open and pushes a sequence of `data:` events to the client as they become available. The response carries `Content-Type: text/event-stream`, and the client consumes it via the browser's `EventSource` API or any HTTP library that supports chunked reads. For an LLM API, the value is direct. Token generation is sequential and slow on a per-token basis, but the first token is usable on its own. Streaming via SSE lets the server flush each generated token as soon as the decode step completes, so the user-visible latency collapses from end to end response time to TTFT. The rest of the tokens arrive at the steady-state decode rate (`1 / TPOT`), but the experience already feels responsive. ```text HTTP/1.1 200 OK Content-Type: text/event-stream data: {"choices":[{"delta":{"content":"The"}}]} data: {"choices":[{"delta":{"content":" answer"}}]} data: [DONE] ``` SSE wins over alternatives for this exact shape. Polling burns RTTs and misses tokens. WebSockets are bidirectional and require an HTTP upgrade handshake plus full-duplex framing the chat case does not need. SSE rides on plain HTTP/1.1 or HTTP/2, traverses corporate proxies and CDNs without special handling, and reconnects via `Last-Event-ID` semantics. OpenAI's Chat Completions, Anthropic Messages, vLLM, and Hugging Face TGI all expose SSE as the default streaming transport for these reasons.

Quick reference

TL;DR

SSE is a one-way HTTP streaming protocol; chat APIs use it so each generated token can be flushed to the client the moment it is produced.

Memory aid

Easy to grasp

Imagine watching a slow printer dispense a long receipt. Instead of waiting for the whole receipt and then handing it to you, the printer slides each line out as soon as the ink dries. SSE is that printer for an LLM. The server opens one ordinary HTTP connection to your browser and keeps it open. Every time the model generates a new token, the server pushes a tiny event down that pipe, and the browser stitches the events together on screen. Compared to alternatives, SSE is simple: plain HTTP, no upgrade handshake, no two-way channel needed because the server is the only one talking. Your code just opens the URL, listens, and renders text as it arrives.

Key concepts

Streaming TTFT Decoding LLM Serving

Detailed answer & concept explanation

~5 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

5 min: define SSE as one-way HTTP streaming, name the `text/event-stream` content type and `EventSource` API, explain the TTFT win for token generation, contrast with WebSockets on directionality and handshake, and name one production gotcha around proxy buffering.

Real-world examples

Real products, models, and research that use this idea.

OpenAI Chat Completions with `stream: true` returns `text/event-stream` and pushes `data: {delta}` chunks plus a final `data: [DONE]` sentinel.
Anthropic Messages streaming sends typed SSE events like `message_start`, `content_block_delta`, and `message_stop` for structured client parsing.
vLLM and Hugging Face TGI both expose OpenAI-compatible SSE endpoints so any OpenAI SDK can consume their streams unchanged.
Production deployments behind nginx or Cloudflare set `proxy_buffering off` and `X-Accel-Buffering: no` to stop the proxy from coalescing the stream.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does the server detect a client disconnect during an SSE stream, and why does that matter for cost?

QWhy does HTTP/2 not replace SSE with server-push for LLM streaming?

QWhat happens to a long-running SSE response if the corporate proxy enforces a 60-second HTTP timeout?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Thinking SSE is bidirectional like WebSockets. SSE is server to client only over plain HTTP; the client uses a separate POST for the request that opened the stream.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.