What's different about prompting a vision-language model (GPT-4o, Claude with image input) versus a text-only model?

**Vision-language models** are LLMs that can read images alongside text. Examples in 2026: GPT-5.5 and GPT-4o with image input, Claude Opus 4.7, Gemini 3.1 Pro. They look like chat models, but prompting them has three twists you do not see with text-only. First, where you put the image relative to the text matters, and the right answer is not the same for every model. Some models do better if you describe the task first and then attach the image. Others handle image-first fine. You have to evaluate for your task. Second, image resolution is not free. Each image is converted into a stack of tokens by tile-based encoding; a higher-resolution image becomes more tokens, costs more, and lets the model see more detail. A small thumbnail might cost a few hundred tokens; a large screenshot can cost a few thousand. Third, if you pass more than one image, label them. 'Image 1 is the original, Image 2 is the edit' lets the model reference them unambiguously. Without labels, asking 'is the third one cropped correctly' is ambiguous.

**Three things change** when you move from text-only to vision-language prompting. First, **image position** is a real prompt-engineering knob. Different providers and even different model versions handle text-first vs image-first ordering differently. Anthropic historically recommends putting images before the question that references them; OpenAI works fine either way for many tasks but exhibits per-task preferences. You evaluate the order on your own dataset rather than assume. Second, **image resolution drives both cost and quality**. Providers tokenize images into a tile-based budget; a higher-resolution image becomes more image tokens, which raises cost and lets the model see fine detail. OpenAI exposes explicit 'low' vs 'high' detail levels with different token costs. Anthropic computes tokens from pixel-tile counts. The implication is that you should match resolution to the task: thumbnails for 'what is this' coarse classification, full resolution for fine-grained reading like dashboards or medical imagery. Third, **multi-image prompts need explicit labeling**. Pass three images without labels and any later reference is ambiguous. Pass them as 'Image 1: original, Image 2: 3 days later, Image 3: 7 days later' and the model can answer questions about each one by reference.