Why does streaming (SSE) improve perceived latency without changing total decode time?

When an LLM generates a reply, it produces one token at a time. A 1000-token answer at 50 tokens per second takes about 20 seconds to finish, no matter how you send it. Without **streaming**, the server waits for all 1000 tokens, then sends the whole reply in one HTTP response. The user stares at a blank screen for the full 20 seconds, then everything appears at once. With streaming, the server sends each token the instant it is generated. The first word appears after time-to-first-token, often just a few hundred milliseconds. The rest of the text flows in steadily as the model keeps generating. The total time to finish is the same in both cases. What changes is when the user first sees something. Perceived latency is the gap between hitting send and seeing a response, and streaming collapses that gap from 20 seconds down to a fraction of a second. **Server-Sent Events** is the standard way to do this over HTTP. It is a simple one-way channel where the server pushes a stream of events to the browser, which is exactly the shape of token by token output.

**Streaming** does not make the model generate any faster. A 1000-token reply at 50 tokens per second still takes about 20 seconds end to end. Total decode time and throughput are unchanged. What changes is perceived latency. This is the time between the user action and the first visible content. Without streaming, the server buffers every token and sends one HTTP body at the end. The user waits the full 20 seconds before seeing anything. With streaming, each token is pushed as it is produced. The first token arrives at time-to-first-token, typically a few hundred milliseconds. Perceived latency drops by one to two orders of magnitude even though nothing about the generation got faster. **Server-Sent Events** is the standard transport. It is a one-way server-to-client stream over a single long-lived HTTP connection. The cost is modest. Each chunk carries small framing overhead. The server must hold the connection open for the whole completion. Both are cheap relative to the large UX gain, which is why every major chat product streams.