Someone confuses 'sequence length' with 'context window'. How do you correct them?
Sequence length is the live token count of this request right now; context window is the model's hard ceiling that never changes. Cost and memory scale with the live count, not the advertised maximum.
Think of a parking lot. The context window is how many cars the lot can hold; it is a fixed property of the lot. The sequence length is how many cars are parked right now; it changes as cars arrive and stops going up when the lot is full. The model's context window is the lot capacity. The sequence length of a specific request is the current car count. People confuse the two because vendors quote the lot's capacity and forget that any given request has its own current count.
Detailed answer & concept explanation~5 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
3 min: define sequence length as a per-request live count, context window as the model's hard ceiling, name the config fields, anchor the 2026 numbers, and close on how cost and KV memory scale with the live count not the advertised window.
Real products, models, and research that use this idea.
- OpenAI usage responses include prompt_tokens, completion_tokens, and total_tokens; total_tokens is the final sequence length for that request.
- Anthropic's client.messages.count_tokens reports the input sequence length before a Claude call so you can budget against the 1M context.
- vLLM logs max_model_len (context window) at startup and per-request prompt_tokens plus output_tokens (sequence length).
- HuggingFace tokenizer.model_max_length exposes the context window; len(input_ids) is the prompt sequence length.
What an interviewer would ask next. Try answering before peeking at the approach.
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Saying a 128K model means every request uses 128K tokens. The 128K is the ceiling. A specific request might use 800 tokens or 80,000; the sequence length is per request and grows during generation.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.