Design slide deck Q&A that cites the right slide without blowing the token budget

Short answer·Medium·4.0 · 0·~3 min·Asked atBain Baseten Promptlayer

Attempt it

Product wants users to upload slide decks and ask questions, with every answer citing the slide it came from. Decks are visually dense: diagrams, charts, screenshots, sparse text. Lay out the key design decisions and the tradeoffs behind each.

Free · 2 AI evals / day

Slide deck Q&A is a multimodal retrieval problem dressed up as a chat product. The mental model: do expensive work once at upload, cheap work at question time. **At upload:** render every slide to an image, run a multimodal embedding model over each, extract any text (slide title, body, speaker notes) for hybrid search, and store everything in a per-deck index keyed by slide number. This is the only time you pay the full cost of the deck. **At question time:** embed the user question, search the index, get back the top three to five candidate slides. Send only those slides to the VLM as images, at high detail so small chart labels are legible. Ask the model to answer the question using the provided slides and name the slide numbers it used. **Citations are free** because the retrieval step already knows which slide each image came from. The model just echoes the number. **Cost stays bounded** because you never send the whole deck. A 60-slide deck would cost 50k-plus tokens at high detail; sending three slides costs a few thousand. The cheap upload-time embedding pays for itself after a handful of questions.

**Six design decisions** that compound into a working slide-deck Q&A product. **Slides as images, not text.** Slide meaning lives in diagrams, charts, and screenshots that text extraction destroys. Keep extracted text alongside images as a cheap retrieval signal, but the model reasons over the rendered slide. **Upload-time indexing.** When a deck is uploaded, render each slide, compute a multimodal embedding (SigLIP-family or a provider's embedding API), extract OCR text, and store both in a per-deck index keyed by slide number. This is the only expensive pass; everything downstream is cheap. **Hybrid retrieval at question time.** Embed the question and search both the image embedding index and the OCR-text index, then fuse scores. Image embeddings catch chart-shaped or diagram-shaped questions; text catches keyword-anchored ones. Return the top 3 to 5 slides. **Send few slides, not the deck.** A 60-slide deck at high detail is roughly 60k tokens. The relevant slide gets lost in the noise even if you can afford the cost. Sending 3 slides keeps the prompt under 5k tokens and the answer stays grounded. **Resolution stays high on the few retrieved slides.** Chart labels and footnotes are small; low detail mode makes them unreadable. Pay the per-image token cost for the slides that matter. **Citations come from retrieval metadata.** The model is told 'these images are slides 14, 22, and 41' and asked to cite which slides it used. Format the citation in the response and your UI links directly to the slide.

TL;DR

Index slide images once at upload, retrieve a few candidates per question, send those few slides at high detail to the VLM, and let citations fall out of the slide-number metadata.

Memory aid

Easy to grasp

Imagine someone hands you a 60-page picture book and asks you a question about it. You do not read every page aloud every time someone asks. Instead, the first time you see the book you flip through it once, take notes on what each page is about, and write the page number on every note. Later, when a question comes in, you check your notes to find the two or three pages that might answer it and only look at those pages carefully. The answer comes with a page number for free. That is the whole design. Slides are pictures, retrieval finds the right pictures, the model only looks at a few of them.

Key concepts

Multimodal AI

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Slide-deck Q&A is the canonical multimodal RAG product. Users upload a deck, ask questions, and expect grounded answers with slide-level citations. The deck is visually dense: diagrams, charts, screenshots, sparse text laid out for visual scanning rather than reading. A text-only RAG approach throws away the meaning that lives in the visuals. A naive vision approach (send the whole deck on every question) explodes cost and buries the relevant slide in noise.

The right design is a two-pass pipeline: expensive once at upload, cheap on every question. Retrieval narrows the model's attention to a few candidate slides; citations fall out of retrieval metadata; cost stays linear in deck size on upload and constant per question. This walkthrough lays out the architecture and the tradeoffs at each layer.

Mental model: the deck is a small visual library. Build the catalogue card once; look up the catalogue every time someone asks a question.

Upload pipeline: index once, cache forever

When a deck arrives, the system does its only expensive pass. The output is a per-deck index that every future question hits.

Render each slide to an image at near-native resolution. Long edge 1280 to 1600 px is a good default: large enough to preserve chart labels, small enough to fit comfortably in provider tile limits. Use PNG to avoid JPEG artifacts on diagrams.

Compute a multimodal embedding per slide. Options in 2026 include Voyage Multimodal 3, Cohere Embed v4, OpenAI's text-embedding-3-large with image extension, or open-source SigLIP-family encoders. The embedding lets you search slides by visual content, not just text.

Extract text via the VLM. A one-shot OCR pass per slide produces title, body text, speaker notes, and any in-image text. Store this text both for hybrid retrieval and for cheap summary answers that do not require sending the image back.

Index by (deck_id, slide_number). The slide number is the citation. Every retrieval result carries it; the model never has to invent it.

Cache aggressively. If the deck is re-opened, re-indexing is free. If the deck is shared across users, the index is shared. Upload cost is paid once per deck, not once per user.

Query pipeline: retrieve few, send few

Cost analysis and amortisation

Evaluation and operational hardening

Production realities and 2026 model lineup

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

ColPali and ColQwen demonstrate per-page image-embedding retrieval over PDFs and decks, dramatically outperforming text-only baselines on visual documents.
Anthropic's Claude Sonnet 4.6 with vision is the typical answer model for this design, with citations grounded to slide numbers.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you handle cross-slide questions like 'compare slide 14 and slide 22'?

Either user pins both slides explicitly, or retrieval expansion: if the question references slide numbers, force-include them; if it asks to compare, retrieve top-k and let the model reason across them.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Sending the whole deck as images on every question. A 60-slide deck at high detail is 50k-plus tokens before the question even starts, blowing latency and cost and burying the relevant slide in noise.

60 second bullets to scan on the way to the call.

Why slide images beat extracted text as the source of truth
The upload-time versus query-time work split

Primary sources. Browse if you want the original framing.