Flashcard: what is a token in LLM prompting and why does it matter for cost and context? · Zenaique

Attempt it

A token is the unit of text the model reads and writes. It is not a word and not a character. The tokenizer splits text into subword pieces, and 1 token is roughly 0.75 English words on average (so 100 tokens cover around 75 words of normal prose). Tokens matter for two practical reasons. First, APIs price by tokens: you pay separately for input tokens (your prompt) and output tokens (the model's reply), and output tokens usually cost 3 to 5 times more. Second, every model has a context-window limit measured in tokens (Claude Opus 4.7, Gemini 3.1, and GPT-5.5 all sit around 1M tokens via API), and that budget covers system prompt plus user message plus history plus retrieved context plus the generated reply. Punctuation, numbers, code, and non-English scripts tokenize denser than plain English, so a 500-word Hindi or Mandarin paragraph can use 2 to 3 times more tokens than a 500-word English paragraph. Word count is never an accurate proxy for token count.

Your answer

Quick reference

TL;DR

A token is the model's subword unit of text, roughly 0.75 English words, and it is the unit both API pricing and context-window limits are measured in.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine a typewriter that can only press one of fifty thousand pre-cut stamps instead of individual letters. Each stamp is a token. Some stamps are whole common words like the or apple, others are partial pieces like tion or ing, and others are single characters for rare symbols. To write any sentence the typewriter has to find the right sequence of stamps. When you ask an LLM a question, the model first turns your text into a sequence of these stamps, then predicts the next stamp, then the next, then the next, until it stops. The price you pay is per stamp pressed, both for reading your input and writing its reply. The maximum number of stamps the typewriter can hold in memory at once is the context window.

Key concepts

Tokenization Context window LLM pricing

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

6 min: define tokens as subword units, give the 0.75-words conversion, separate input vs output pricing, explain the context-window budget, name the non-English and code penalty, point to tokenizer libraries.

Real-world examples

Real products, models, and research that use this idea.

Anthropic publishes a tokenizer endpoint so teams can count tokens exactly before sending a Claude Opus 4.7 request.
OpenAI's tiktoken library is the standard way to count tokens locally for GPT-5.5 and GPT-4o prompts without making a network call.
Google's Gemini 3.1 docs show that a 500-page novel fits in about 700K tokens, well inside the 1M context window.
The Llama 3 family uses a 128K-token vocabulary via a tiktoken-style BPE tokenizer (Llama 2 used SentencePiece), roughly double the older vocab size, partly to improve tokenization efficiency on code and non-English text.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy are output tokens typically 3 to 5 times more expensive than input tokens?

Sign in or go Pro to see the approach for this follow-up.

QHow does prompt caching change the token economics of a multi-turn conversation?

Sign in or go Pro to see the approach for this follow-up.

QWhy was early LLM arithmetic so bad, and how does tokenization explain it?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Estimating token count by multiplying word count by 1.0 and being surprised when a code-heavy or non-English prompt blows the context-window budget; tokens are not words.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.