Explain how vision-language APIs price image inputs, what determines the per image token count, and why a hi res photo can outweigh several pages of text on the input token bill.
Vision encoders tile each image into fixed patches and emit hundreds of tokens per tile, all billed at the standard input rate.
Picture a postcard rack at a store. Text prompts are like letters, the cashier counts each word. Images are not letters, so the store cuts each photo into small squares and counts the squares. A small photo is one or two squares; a giant printed poster is many dozens. You pay per square just like you pay per word. A poster can easily cost more than a long letter because it has so many squares. The store offers a thumbnail option: print a tiny version that is only one square, much cheaper, but you lose detail. The trick is matching the photo size to how much detail the cashier actually needs to do their job.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Vision-language models bill images as tokens, not as images. Sending a single photo can cost more input tokens than a multi-paragraph prompt, and teams ship features oblivious to that until the API bill spikes. The mechanism is straightforward once you see it: a vision encoder tiles the image, the projector emits a fixed number of tokens per tile, and every one of those tokens is billed at the standard input rate.
This deep dive walks through the encoder architecture, the tile-grid math that determines per-image token count, the detail or resolution parameter that bounds it, and the operational mistakes that quietly multiply the bill. The same mechanics extend to video, where the cost compounds by frame.
What happens between the image upload and the LLM
A vision-language model has two main parts wired together. The first is a vision encoder, typically a CLIP-style or SigLIP-style vision transformer pretrained on a massive image-text contrastive task. The second is the language model itself, a standard decoder-only transformer. Between them is a projector: a small adapter, often a couple of linear layers or a Q-Former, that maps the visual encoder's patch embeddings into the language model's token-embedding space.
When you upload an image, the API does three things. First, it preprocesses the image: resize, normalize, tile. Second, it runs the vision encoder on each tile and produces a fixed-size embedding sequence per tile. Third, the projector compresses or reshapes that sequence into a fixed count of LLM tokens per tile, which get spliced into the prompt at the position where the image appears.
The language model then attends to those image tokens the same way it attends to text tokens. They are not special data types from the LLM's perspective; they are vectors in the same embedding space as the word tokens, surrounded by text tokens describing the user's question. The whole sequence flows through the standard transformer stack. This is why image tokens count against the same input-token budget and are billed at the same per-token rate.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Image setup | Tiles | Approx tokens | Good for | Bad for |
|---|---|---|---|---|
| 512x512 low detail | 1 overview only | ~85 | scene classification, rough description | OCR, fine text, small icons |
| 512x512 high detail | 1 tile + overview | ~255 | small photos, simple icons | high-resolution sources downscaled and losing detail |
| 1024x1024 high detail | 4 tiles + overview | ~765 | typical product photos, charts | fast turnaround on bulk image jobs |
| 2048x2048 high detail | 16 tiles + overview | ~2805 | detailed documents, dense charts | any task where low detail suffices |
| 4K photo high detail | 32-48 tiles + overview | ~5000+ | rare cases needing full resolution | default uploads from phone clients |
Real products, models, and research that use this idea.
- OpenAI's vision pricing on GPT-5.5 uses 170 tokens per 512x512 high-detail tile plus an 85-token overview; a 1024x1024 image at high detail is 765 input tokens.
- Anthropic's Claude Opus 4.7 vision API charges per image with high-resolution and low-resolution modes; a single hi-res photo commonly lands at 1500-2500 input tokens.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does aspect ratio interact with the tile grid to inflate token cost?
Tiles are fixed 512x512 squares, so an image's tile count is the ceiling of its dimensions divided by 512 in each axis. A tall thin screenshot may pay for a 2x6 grid even though most of each row tile is empty. Pre-cropping to the content region often cuts tile count by 30-50 percent on document images.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Assuming image inputs are free or flat-priced. Every image is decomposed into image tokens billed at the standard input rate; a hi-res photo can quietly outprice the entire surrounding text prompt.
60 second bullets to scan on the way to the call.
Explain how a vision encoder converts an image into tokens before the LLM sees it.
Describe tile-based encoding and the role of fixed-resolution tiles.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.