What is the EOS token and how does the runtime use it?

Attempt it

Your answer

Quick reference

TL;DR

EOS is a special vocabulary token the model is trained to emit when its answer is complete; the runtime sees it, halts generation, and reports finish_reason=stop.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Picture someone telling a story out loud. They need a way to signal they are done so the listener does not just wait forever. They might say the word period or just go quiet at a clear ending point. The model has the same problem. It writes one word at a time and needs a signal that means stop here. During training it learns one special invisible word for exactly that purpose. The serving system watches the stream of tokens, and the moment that special stop word appears, it ends the reply right away. The word never gets shown to the user. Without that signal the model would keep generating until some other limit kicked in, which would feel sloppy.

Key concepts

EOS token Stop sequence Max tokens cap Tokenizer

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

4 min: define EOS as a vocabulary token the model emits at response end, walk through the sampler-halt mechanic, contrast with max-tokens and stop strings via finish_reason, then cover the multi-EOS reality of modern chat models and how fine-tunes break it.

Real-world examples

Real products, models, and research that use this idea.

Llama 4 Maverick uses `<|eot_id|>` and `<|end_of_text|>` as role-specific EOS tokens, both registered as stop ids in vLLM and TGI.
Qwen 3.5 chat models stop on `<|im_end|>`, which marks the end of each assistant turn in the ChatML format.
OpenAI's GPT-5.5 API returns finish_reason=stop on natural EOS and finish_reason=length when the max-tokens budget cuts the model off.
Anthropic's Claude Opus 4.7 streams a stop_reason field that distinguishes end_turn (EOS), max_tokens, and stop_sequence (user-supplied).
Gemma 4 instruction-tuned models terminate on the `<end_of_turn>` token, which the open-weight community had to discover the hard way during early fine-tunes.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy do modern chat models use multiple end tokens instead of just one EOS?

Sign in or go Pro to see the approach for this follow-up.

QHow would you debug a fine-tuned model that keeps generating past where it should stop?

Sign in or go Pro to see the approach for this follow-up.

QWhat does the runtime do with logits for EOS during sampling?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Confusing EOS with a max-tokens limit or a user-supplied stop string. Only EOS reflects the model's own decision; the other two are external policy.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.