Bytes, characters, tokens: which one gets billed and how do they relate?

Flashcard·Easy·4.0 · 0·~30s

Attempt it

Billing is per token, never per byte or per character. A token is a tokenizer-defined unit produced by a BPE (or similar) merger over UTF-8 bytes. For English prose, the ratio sits around 3 to 4 characters per token, so a 1 KB English prompt is roughly 250 tokens; for code, JSON, or non-Latin scripts the ratio shifts and a single character can cost several tokens. There is no fixed mapping from bytes or characters to tokens, because the tokenizer is content-dependent and model-specific (cl100k_base for GPT-4, tiktoken o200k_base for GPT-4o, SentencePiece variants for Llama). The only reliable way to count is to run the model's tokenizer over the exact string.

Your answer

**LLM APIs bill per token**, never per byte or per character. A token is a unit the model's tokenizer produces by splitting your text into chunks, then mapping each chunk to an integer in the model's vocabulary. The same text, run through different tokenizers, will produce different token counts. For English prose, a useful rule of thumb is about 3 to 4 characters per token. A typical English word is one or two tokens. A 1,000-character English email is roughly 250 tokens. This is why providers can give back of the envelope estimates like '1 page is about 500 tokens' for natural-language prose. The ratio breaks down when the content is not English prose. Code uses lots of punctuation, identifiers, and indentation that the tokenizer was not trained to compress well; a 1,000-character Python file might tokenize to 350 tokens or more. Minified JSON has the same problem. Non-Latin scripts (Hindi, Arabic, Japanese, Chinese) tokenize much worse: a single character might consume several tokens because the underlying UTF-8 bytes get split into small pieces. The only reliable way to count tokens is to run the model's own tokenizer over your exact string. OpenAI provides the `tiktoken` library; Anthropic provides a `count_tokens` API; Hugging Face Transformers exposes any tokenizer via `AutoTokenizer`. Treat any character-based estimate as a rough guide, never as a billing prediction.

**LLM APIs bill on tokens**, and the byte to token ratio is a property of the tokenizer and the content, not a fixed conversion. Every modern LLM uses a BPE-family tokenizer (byte-pair encoding or a close variant like SentencePiece or WordPiece), trained on a specific corpus to greedily merge frequently-occurring byte sequences into vocabulary entries. The practical implications fall out from how BPE training works. Common English words and word-pieces in the training corpus get their own tokens, so English compresses well: roughly 3 to 4 characters per token, or about 250 tokens per 1,000 characters. Less-common patterns decompose into multiple tokens. The result is that token count depends on what you wrote, not just how long it is. Three examples drive this home: - **English prose (1,000 chars)**: ~250 tokens. - **Minified JSON (1,000 chars)**: ~350-450 tokens. Quotes, braces, and short keys do not compress. - **Hindi or Japanese text (1,000 chars)**: ~700-1,200 tokens. The tokenizer was trained on mostly-English text, so non-Latin characters often decompose into raw UTF-8 byte tokens. Different models use different tokenizers. `cl100k_base` is GPT-4 / GPT-3.5 / text-embedding-3. `o200k_base` is GPT-4o / GPT-4o-mini / o1 / o3, with a larger vocabulary (200k vs 100k) that compresses better for code and non-English. Llama 3 uses a 128k-token SentencePiece vocabulary. Claude 3 / 4 use their own proprietary tokenizer. The same string can have very different token counts across these models. The operational implication is that you cannot estimate cost reliably from character or byte count alone. The OpenAI tiktoken library, Anthropic's `count_tokens` endpoint, and Hugging Face's `AutoTokenizer.encode()` are the only authoritative count sources. Production cost models always run the actual tokenizer; estimating from character count introduces 20-40% error on mixed workloads.

Quick reference

TL;DR

Billing is per token, not per byte or character. The byte to token ratio is content-dependent and model-specific; only the model's tokenizer tells you the exact count.

Memory aid

Easy to grasp

Imagine a courier that charges by parcels, not by weight. You arrive with a bag of stuff and the courier breaks it into parcels according to their own rules. English text packs neatly: about 3 to 4 characters per parcel. Minified JSON or Python code does not pack neatly, because the courier's rules were built for prose and weird punctuation breaks at the seams. Hindi or Japanese characters use up parcels at a different rate again, because the rules learned during training were trained on mostly-English text and never saw enough non-Latin characters to compress them well. So 1 KB of English costs about 250 parcels; the same 1 KB of code or non-English text can cost twice as much. The only way to know your real bill is to run the courier's own ruler over your specific bag.

Key concepts

Tokenization Byte-Pair Encoding Cost & Billing Context Window

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

5 min: distinguish bytes, characters, and tokens, state that billing is per token, explain BPE produces content-dependent compression ratios, give numbers for English vs code vs non-Latin text, and name the canonical tools for accurate counting.

Real-world examples

Real products, models, and research that use this idea.

OpenAI's `tiktoken` library counts tokens locally; `tiktoken.encoding_for_model('gpt-4o')` returns the o200k_base encoder used for billing.
Anthropic's `messages.count_tokens` endpoint returns the exact billable token count for a Claude prompt before submitting it.
Llama 3 ships a 128k-vocabulary SentencePiece tokenizer that improves compression on non-English text versus Llama 2's 32k vocabulary.
GPT-4o's switch from cl100k_base to o200k_base cut Hindi token counts by roughly 4x and Korean by ~1.7x, lowering effective cost for those markets.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does the same text cost more tokens in cl100k_base than in o200k_base?

QHow does byte-level BPE prevent out of vocabulary issues?

QWhy is changing a model's tokenizer disruptive even if the model architecture stays the same?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Estimating cost by character count. Tokenizers compress differently across content types, so character-based estimates can be off by 2x or more for code and non-Latin text.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.