What can GraphRAG answer that top-k vector retrieval structurally cannot, and when is the cost justified?

Short answer·Hard·4.0 · 0·~3 min

Attempt it

Describe what GraphRAG does differently from standard vector RAG, the class of questions it unlocks, and the conditions under which its higher build cost is worth paying.

Free · 2 AI evals / day

Standard RAG retrieves the top-k chunks most similar to your query and feeds them to the LLM. That is a local view of the corpus — only the passages nearest your query in meaning. That works for fact lookups. It fails for global questions like "what are the main themes across all these documents?" because no single chunk holds that answer, and raising k just adds more nearby chunks, not a view of the whole corpus. **GraphRAG** does extra offline work to fix this. An LLM reads the corpus, extracts entities and relationships into a knowledge graph, clusters that graph into communities, and writes a summary for each community. Those summaries together cover the entire corpus. At query time, a global question is answered by combining the community summaries instead of raw chunks, so the answer reflects the whole collection. The cost is that building and refreshing the graph uses a lot of LLM tokens. It is worth it when users ask big-picture, sensemaking questions over a connected, fairly stable corpus. It is overkill when they only ask simple fact lookups that plain vector RAG already handles cheaply.

The structural difference is where synthesis happens. Vanilla RAG retrieves the top-k locally similar chunks and reasons over them at query time, so it can only ever answer from a local slice. GraphRAG moves synthesis to index time. An LLM extracts entities and relationships into a graph, communities are detected, and each community is summarized. That builds a multi-resolution digest of the entire corpus before any query arrives. The unlocked class is global, aggregative questions. Examples include recurring themes, how a narrative shifted, and what connects a set of incidents. These have no answer in any single chunk, so no value of k recovers them. GraphRAG answers them by map-reducing over community summaries. Each summary yields a partial answer, and the partials are combined into a final response grounded in the whole corpus. The cost justification is the senior part of the answer. Extraction and summarization run every chunk and every community through an LLM, which is dollars per million source tokens and hours of build, and refresh is painful when documents churn. So the test is question mix and corpus stability. Justified: connected, relatively stable corpora where sensemaking questions dominate — incident reports, research collections, legal discovery. Overkill: fact-lookup workloads or fast-changing feeds, where cheaper options like query-focused summarization or simply paginating the corpus through a long-context model win. GraphRAG is a targeted tool, not a default upgrade.

Quick reference

TL;DR

GraphRAG pre-builds an entity graph, communities, and summaries so global "sensemaking" questions can map-reduce over the whole corpus — worth it only when those questions dominate a connected, stable corpus.

Memory aid

Easy to grasp

Imagine an analyst handed a whole filing cabinet and asked "what story do all these files tell together?" Plain search just pulls the few folders whose labels match your words — no help for a question about the whole cabinet. GraphRAG hires the analyst ahead of time: they read everything, map how the files connect, group them into related stacks, and write a one-page summary per stack. Later, the big-picture question gets answered by reading those summaries and stitching them together. The honest catch is that hiring the analyst to read the entire cabinet is expensive, and every time files change you pay them to re-read. So you only do it when people keep asking whole-cabinet questions, not when they just want one fact from one folder.

Key concepts

Retrieval-Augmented Generation LLM System Design Agents

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

5-6 min: the local-operator limit of top-k, index-time versus query-time synthesis, the build economics, refresh as the operational killer, and the alternatives that change the build or skip decision.

Real-world examples

Real products, models, and research that use this idea.

Microsoft GraphRAG applied to research-corpus sensemaking, the use case its original write-up benchmarks.
Incident-retrospective tooling that asks "what patterns recur across last quarter's outages" over a connected report set.
Legal e-discovery where the question is about relationships and themes spanning thousands of connected documents.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhen would a hierarchical summary tree beat full GraphRAG for global questions?

QHow do you keep a GraphRAG graph fresh on a corpus that changes daily?

QHow would you evaluate whether GraphRAG actually beats vanilla RAG on your global questions?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Recommending GraphRAG as a default upgrade without sizing the build cost. On a churning corpus or a fact-lookup workload it burns tokens for capability nobody uses — the cheaper move is often query-focused summarization or long-context.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.