Why images are expensive to process in a VLM, and the core tradeoff that creates

Short answer·Medium·4.0 · 0·~3 min·Asked atBytedance Droom Hcl

Attempt it

Explain why feeding images to a vision-language model costs far more than feeding the same query as text, and name the central tradeoff a team has to manage because of it.

Free · 2 AI evals / day

Images are expensive because **one image is not one token**. The vision encoder turns an image into a block of hundreds of vision tokens, and the language model has to process all of them. High resolution makes it worse. To read fine detail, modern VLMs use tiling: they split the image into a grid of tiles, encode each tile separately, and add a downscaled global view. Each tile is its own block of tokens, so a high-resolution image can become thousands of tokens. Those vision tokens go through the model's prefill exactly like text tokens. Prefill cost grows with how many tokens are in the prompt, so a big image dominates both the prompt size and the time before the model starts answering. A typed question might be 30 tokens; a tiled image might be 2,000. The central tradeoff is **resolution**. More resolution and more tiles buy accuracy on small text, dense charts, and fine detail, but they cost more latency and money on every single request. The practical move is to set a token budget per use case: feed low resolution for simple scene photos and reserve high-resolution tiling for documents and charts where the fine detail actually matters.

Feeding an image costs far more than the same query as text because the image expands into many tokens before the model ever reasons over it. The encoder converts an image into a block of vision tokens, typically hundreds for a single base-resolution pass. High-resolution **tiling** multiplies that: the image is split into a grid of tiles, each tile emits its own block of tokens, and a downscaled global view adds one more block. So a single high-resolution image can run to thousands of tokens, while the user's typed question is a few dozen. Those vision tokens are not cheap metadata. They flow through prefill just like text, and attention cost grows with sequence length, so the image dominates both the prompt size and the prefill latency. The image, not the text, usually decides time to first token. That sets up the central tradeoff: **resolution against cost**. More resolution and more tiles buy accuracy on small text, dense charts, and fine detail, because each region gets a higher-fidelity representation. But every added tile is a full block of tokens charged on every request, in both dollars and latency. The senior move is to set an image-token budget per use case rather than always maxing resolution. A simple scene photo needs far fewer tokens than a dense document scan. Matching the tiling level to the content is how teams keep cost in line with the accuracy the task actually requires.

TL;DR

One image becomes hundreds to thousands of vision tokens that go through prefill like text, so the central tradeoff is resolution: more tiles buy fine-detail accuracy but cost latency and money every request.

Memory aid

Easy to grasp

Imagine you pay a reader per page they read. A short typed question is a single page, quick and cheap. But an uploaded photo is not one page. The model slices it into many little pieces and, if you ask for high detail, slices it into even more pieces so it can read the fine print. Suddenly that one photo is hundreds or thousands of pages, far more than your typed question. The reader has to flip through every page before answering, which takes time and costs money. So the big decision is how finely to slice each image. Slice it coarsely and you save pages but might miss small text. Slice it finely and you catch the detail but pay for all those extra pages.

Key concepts

Multimodal AI Inference Optimization Tokenization

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Engineers new to multimodal work tend to picture an image slotting into a prompt as cheaply as a sentence. The reality is the opposite, and this question exists to surface that gap. The answer the interviewer wants connects a mechanism (how an image becomes tokens) to a production decision (how to budget resolution).

The reason this is a tradeoff and not a fact to memorize is that both sides genuinely bite. Low resolution loses real information that some tasks depend on. High resolution costs real latency and money on every request. There is no setting that is correct for all inputs, which is exactly why it is an engineering decision.

This walkthrough quantifies why an image is expensive, traces the cost past the price sheet into prefill and the KV cache, frames the resolution tradeoff honestly with its diminishing returns, and lays out the budgeting and routing patterns that teams use to manage it. The thread throughout is that resolution is a cost knob, and senior engineers tune it on purpose.

Why one image dwarfs a text prompt

Start with the asymmetry. Text arrives already tokenized: a typed question of a dozen words is maybe 30 tokens, and the model attends over them directly. An image has no native tokens. The vision encoder has to manufacture them by patchifying the image and emitting a fixed block, typically a few hundred vision tokens for one base-resolution pass.

Then tiling multiplies it. A single encoder pass downscales the image, which throws away fine detail. To read small text and dense charts, modern VLMs tile: they split the image into a grid, encode each tile separately, and add a downscaled global view. Each tile is its own block of a few hundred tokens. A high-resolution document tiled into a 3x3 grid plus a global view is ten blocks, easily two to three thousand tokens.

Line the two up. The user types a 30-token question and attaches one document image worth 2,560 tokens. The image is roughly 85x the cost of the text it accompanies. That ratio is the heart of the question. Anyone who answers that an image costs about the same as a short prompt has the wrong mental model by two orders of magnitude, and that is the single most important thing to get right here.

The cost is prefill and memory, not just price

The resolution tradeoff, honestly

Budgeting and routing in practice

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

OpenAI GPT-5.5 high-detail images are priced as a base block plus per-tile blocks, so a full-page scan costs many times a thumbnail.
LLaVA-NeXT AnyRes tiling expands a high-resolution image into multiple per-tile token blocks plus a global view, the canonical open-source version of this cost.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does the resolution versus cost relationship scale roughly with the square of resolution?

Tiling covers a 2D area. Doubling linear resolution roughly quadruples the number of fixed-size tiles needed, so the per-tile token blocks, and the cost, quadruple too.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Thinking an image costs about the same as a short text prompt. One image routinely costs hundreds to thousands of tokens, dwarfing the user's typed query.

60 second bullets to scan on the way to the call.

Why does one image cost hundreds to thousands of tokens while a text query costs a few dozen?
How does tiling multiply the token count beyond a single encoder block?

Primary sources. Browse if you want the original framing.