In a chat, 'what about its pricing?' returns junk from retrieval: diagnose the failure and fix it

Short answer·Medium·4.0 · 0·~3 min·Asked atFireworks Ai Pinterest Turbopuffer

Attempt it

Your conversational RAG assistant works fine for standalone questions, but when a user follows up with 'what about its pricing?', retrieval comes back with irrelevant chunks. Explain precisely why retrieval fails on this turn, and describe the fix.

Free · 2 AI evals / day

The assistant takes whatever the user just typed, embeds it, and searches the vector index with that embedding. For a standalone question like 'What does the Pro plan cost?' that works, because the sentence fully describes what the user wants. The follow-up 'what about its pricing?' breaks because of the word **its**. That pronoun points back to something said in an earlier turn — maybe a product, a plan, a competitor. By itself, the sentence carries almost no searchable content. So when the system embeds that short, dependent sentence, the resulting query vector points at a vague region of the space that does not describe anything specific. The retriever returns junk, because there is nothing meaningful to match against. The key realization: the retriever only sees the current query string. It has no access to the conversation history where 'its' was defined. The fix is **query contextualization**, also called query rewriting. Before retrieval, run a step that uses the chat history to rewrite the follow-up into a standalone question — 'what about its pricing?' becomes 'What is the pricing of the Enterprise plan?' Now the embedded query actually means something, and retrieval lands on the right chunks.

The failure is a coreference problem at the embedding step. The system embeds the follow-up turn literally — 'what about its pricing?' — and hands that vector to the retriever. But the meaning of that sentence lives in prior turns: **its** is a pronoun referring to an entity named earlier, like a specific product or plan. Stripped of that history, the sentence has almost no retrievable content. So the query vector points at a vague, near-meaningless region of the embedding space, and the retriever pulls back whatever happens to be nearby — junk. The root cause is that retrieval only sees the current query string; it has no window into the conversation where the referent was established. The fix is **query contextualization** — also called query rewriting or condensation. Before retrieval, insert a step that takes the conversation so far plus the new turn and rewrites the follow-up into a standalone, self-contained query. The dependent turn becomes a self-contained one, so a vague follow-up about pricing turns into a full question naming the Enterprise plan. That resolves the coreference, so the embedded query genuinely describes what the user wants and retrieval matches the right chunks. Placement is the detail that decides everything: this rewrite must sit before retrieval, not after. A post-filter or reranker cannot help, because the original query vector already failed to surface anything relevant — there is nothing good in the candidate set to rescue.

TL;DR

The follow-up is embedded literally with 'its' unresolved, so the query vector is near-meaningless. Fix it by rewriting the turn into a standalone query before retrieval — query contextualization.

Memory aid

Easy to grasp

Imagine texting a librarian who can only read your latest message, never the chat above it. You ask 'What's the population of France?' and get a great answer. Then you text 'what about its capital?' — but the librarian, seeing only that one line, has no idea what 'its' means. So she fetches random books. The problem is she never saw the France message. The fix is to clean up your follow-up before sending it: rewrite 'what about its capital?' into 'What is the capital of France?' so it makes sense on its own. A conversational search assistant works the same way — it has to rewrite a dependent follow-up into a complete question before it searches, because the search only sees the words you hand it.

Key concepts

Retrieval-Augmented Generation Embeddings LLM System Design

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Conversational RAG looks deceptively close to single-turn RAG. You embed the user's question, retrieve, and generate. The gap that bites teams is that human conversation is full of references that only make sense in context — pronouns, ellipsis, 'that one,' 'the second option.' A retriever, by contrast, is stateless: it sees the string you hand it and nothing else.

This question is a clean instance of that mismatch. 'What about its pricing?' is a perfectly natural follow-up, but the word that carries its meaning — 'its' — points at something said earlier, and the retriever never saw that turn. The result is not an error message; it is silent garbage, which is worse, because the system keeps answering, just from the wrong context.

This deep dive nails the exact mechanism of the failure, locates the fix at the precise point in the pipeline where it must go, explains why the popular wrong fixes (reranking, history-stuffing) do not work, and covers the real-world failure modes of the rewriting step so you can talk about it like someone who has shipped it.

The exact mechanism: a pronoun with no referent in the vector

Trace what happens to 'what about its pricing?' Step one, the system embeds the string. Step two, it searches the index for nearest neighbors. Step three, it feeds whatever came back to the generator.

The failure is entirely in step one. An embedding model maps a string to a point in space based on the string's content. 'What about its pricing?' has a real chunk of content — pricing — but the entity that pricing belongs to is hidden behind the pronoun 'its.' The model cannot resolve 'its' because the antecedent is in a previous turn that is not in the string. So the embedding effectively encodes 'something about pricing,' an entity-less query.

An entity-less query lands in a diffuse region of the space. There is no single product, plan, or document that 'something about pricing' is closest to; it is roughly equidistant from every pricing-related chunk in the corpus. Top-k over that point returns whatever happens to be marginally nearest, which is noise relative to what the user meant.

The critical insight is that nothing downstream is broken. The embedding model worked, the vector search worked, the generator worked. The query simply did not contain the information needed to retrieve correctly, because the human relied on conversational context the retriever cannot see.

Why the retriever cannot see the conversation

The fix: query contextualization before retrieval

Why reranking and history-stuffing both fail

Production reality: latency, wrong referents, and detection

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

LangChain's condense-question chain rewrites a follow-up into a standalone query using chat history before retrieval.
LlamaIndex's CondenseQuestion chat engine performs the same rewrite step ahead of the retriever.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhat can go wrong with the rewriting step itself, and how do you guard against it?

The rewriter is an LLM and can bind the pronoun to the wrong entity on ambiguous history, or over-specify and narrow retrieval too far. Discuss passing only the last few turns to limit confusion, low-confidence fallback to the raw query plus history, logging rewrites for evaluation, and the per-turn latency cost of the extra LLM hop.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Trying to fix this by filtering or reranking retrieved chunks. The query vector was already meaningless, so there is nothing good in the candidate set to rerank toward.

60 second bullets to scan on the way to the call.

Diagnose the unresolved pronoun and why it empties the query of content
Explain that the retriever is stateless and sees only the current string

Primary sources. Browse if you want the original framing.