Where should an image be placed relative to the text that asks about it, and why does this matter for multimodal models?

Attempt it

Place the image immediately adjacent to the text that refers to it, with the image first and the question right after. Two reasons. First, the question reads as a continuation of what the model just saw, recency works for vision tokens the same way it works for text. Second, when there are multiple images the model can be confused about which one a later sentence refers to; co-locating each image with its question removes the ambiguity. Across providers the exact tokenization of images differs, but the locality principle is the same: keep the question and its image together, not separated by a wall of other content.

Your answer

The short answer: **place the image immediately before the text that asks about it**, and keep them adjacent. There are two reasons this matters. **Reason 1: recency.** Modern multimodal models (Claude Opus 4.7, GPT-5.5, Gemini 3.1 Pro) attend to nearby content more reliably than to distant content, the same lost-in-the-middle effect that applies to text applies to vision tokens. Putting the image right before the question makes the question read as a natural continuation of what the model just saw. **Reason 2: reference ambiguity.** When you send a single image and a single question, the model can figure out the binding even if the image is at the top and the question is at the bottom. When you send three images and three questions, the binding becomes ambiguous unless each image is co-located with its question. The model might attend to the wrong image when answering, especially if the images look similar. The pattern that works: ``` [image_1] Question about image 1. [image_2] Question about image 2. [image_3] Question about image 3. ``` The pattern that fails: ``` [image_1] [image_2] [image_3] Question about the first image. Question about the second image. Question about the third image. ``` In the second pattern the model has to do extra work to bind 'first' to `image_1`, and it gets it wrong more often than the co-located version. The exact image tokenization differs across providers, but the locality principle is universal. Keep each image next to the text that refers to it.

**The image goes immediately before the text that asks about it, and the two stay adjacent in the prompt.** This is the universal recommendation across 2026 multimodal providers (Anthropic, OpenAI, Google) for a reason that has both a positional component and a reference-resolution component. **The positional component is recency.** Multimodal models are subject to the same kind of positional attention bias that applies to text-only models. Content near the end of the input is attended to more reliably than content in the middle. When the image is placed right before the question, the question reads as a continuation of what the model just saw, the vision tokens are still recent in the attention pattern when the question begins. When the image is placed at the start of the prompt and the question is far below, the model has to look back across a wall of other content to ground its answer, and quality degrades. **The reference-resolution component matters most with multiple images.** When a prompt contains one image and one question, the binding is unambiguous: there is only one image to refer to. When a prompt contains three images and three questions, the binding has to be derived from the text. 'The first image' and 'image 2' work in principle, but the model can attend to the wrong image, especially when the images look similar. Co-locating each image with its question makes the binding **structural rather than referential**, the model does not have to figure out which image goes with which question, because they are adjacent. The pattern that works: ``` [image_1] What type of architecture is shown here? [image_2] What material is the facade made of? [image_3] Approximately what decade is this from? ``` The pattern that fails: ``` [image_1] [image_2] [image_3] For the first image, what type of architecture is shown? For the second image, what material is the facade made of? For the third image, approximately what decade is this from? ``` The second

Quick reference

TL;DR

Put the image immediately before the text that asks about it; co-locating image and question removes reference ambiguity in multi-image prompts and aligns with the recency the model uses for both vision and text tokens.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine showing a friend a photo and asking 'what do you see?' If you show the photo, then ask, the connection is obvious. If you ask first and pull out the photo later, your friend has to remember the question while looking at the picture. Now imagine showing three different photos and asking three different questions in a jumbled order, your friend will get confused about which question goes with which photo. The trick is to pair each photo with its question right next to each other. Models work the same way.

Key concepts

Context Engineering

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

State the rule: image immediately before its question, adjacent. Explain the two rationales: positional recency (lost-in-the-middle applies to vision tokens) and reference resolution (co-location removes ambiguity in multi-image prompts). Walk through the good pattern (interleaved pairs) and the bad pattern (stacked images plus stacked questions). Note that the locality principle is universal across 2026 providers even though image tokenization differs. Extend to video and audio. Close with the engineering rule: treat multimodal content as content that needs placement, not as a header.

Real-world examples

Real products, models, and research that use this idea.

Anthropic's Claude SDK pattern interleaves image and text content blocks for multi-image prompts in production
OpenAI's GPT-5.5 documentation recommends image then question pairing for multi-image visual QA
Medical imaging assistants in 2026 use co-located image plus finding patterns to ground each finding to a specific scan
Document-understanding products in 2026 pair each page image with its per-page question to avoid cross-page reference confusion
Robotics and embodied AI demos in 2026 pair each frame with its time-stamped action prompt to keep grounding tight

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you arrange a prompt that has 20 images and one summarizing question across all of them?

Sign in or go Pro to see the approach for this follow-up.

QWhy does the position issue persist even in 2026 frontier models with very long context windows?

Sign in or go Pro to see the approach for this follow-up.

QHow does this principle interact with prompt caching when the prompt has both static images and a variable question?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Putting all images at the top of the prompt and then asking multiple questions about them in a later block, leaving the model to guess which question maps to which image.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.