Match each RAGAS metric to what it specifically measures about a RAG pipeline.
Drag each answer to line up with its matching prompt
Faithfulness
What fraction of the retrieved chunks are actually relevant to the query (low precision = retrieval brings back noise alongside signal).
Answer relevance
Whether the LLM's answer actually addresses the user's query, independent of whether it's grounded in retrieval.
Context precision
Whether the retrieved chunks collectively contain the information needed to answer the query (low recall = the right chunks weren't retrieved).
Context recall
Whether the LLM's answer is grounded in the retrieved chunks: no contradictions, no fabricated claims not present in the context.
RAGAS splits RAG quality on two axes: the answer (faithfulness = grounded, answer relevance = on-topic) and the context (precision = retrieved chunks are useful, recall = needed chunks were retrieved).
Picture a student writing an open book exam. The librarian fetches some books for them, then they write an answer. Four things can go right or wrong. Did the librarian fetch the right books at all (context recall)? Or pile on junk books with one useful one buried inside (context precision)? Then the student: did they write only what's actually in the books, or invent stuff (faithfulness)? And did they answer the question that was asked, or wander off topic (answer relevance)? RAGAS scores all four separately. That separation is the whole point: a bad final answer could be the librarian's fault or the student's, and you need to know which before you can fix it.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
RAGAS is the most cited answer to a question every RAG team eventually hits: the pipeline gave a bad answer; now whose fault is it, the retriever or the generator? A single end to end accuracy score cannot tell you, because a RAG system is a composition of two independent stages, and either can fail while the other works perfectly. Worse, the two failures can cancel or compound in ways that a blended score smears into meaningless noise.
The framework's core move is to lay quality on a 2x2 grid. One axis is the context (what the retriever returned). The other axis is the answer (what the LLM generated from that context). Each axis gets two metrics, and the whole skill in this question is keeping the four straight and understanding why they are deliberately separate rather than blended into one number. Memorizing the labels is not enough; an interviewer will probe whether you can predict which metric moves when you change a chunk size, swap an embedding model, or tighten a prompt.
This deep dive defines each metric precisely, shows why they are orthogonal, explains which one caps the entire pipeline, walks a concrete failure through all four scores, and closes on how teams operationalize the split in CI and production monitoring.
The 2x2: why four metrics and not one
A RAG pipeline has two failure surfaces stacked in series. The retriever decides which chunks reach the model; the generator decides what to write given those chunks. A bad final answer can originate in either stage, and the failures are independent: a perfect retriever feeding a hallucinating model fails just as hard as a faithful model fed garbage chunks. Because the stages are sequential, a failure upstream silently constrains everything downstream, which is exactly why one number cannot localize the problem.
RAGAS captures this with a two by two. The context axis grades the retriever: context recall and context precision. The answer axis grades the generator: faithfulness and answer relevance. Read across the grid and you get four yes/no questions (did we fetch the right material, did we fetch it cleanly, did we use only that material, and did we actually answer the question) each isolating one degree of freedom in the system.
The reason this matters is attribution. If you only logged one blended quality number and it dropped, you would have no idea where to look. With the split, a recall drop sends you to the index and chunking; a precision drop sends you to ranking; a faithfulness drop sends you to the prompt and model; a relevance drop sends you to query understanding. Collapsing the four back into a single average throws away exactly the diagnostic signal the framework exists to produce, which is why reporting one blended RAGAS score is a classic anti-pattern that quietly hides regressions. Treat the four as a dashboard, not a leaderboard.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Metric | Axis | What it measures | Low score points at |
|---|---|---|---|
| Faithfulness | Answer | Every claim is supported by the retrieved context | LLM hallucinating beyond the context |
| Answer relevance | Answer | The answer addresses the user's actual query | Evasive, padded, or off-topic generation |
| Context precision | Context | Fraction of retrieved chunks that are relevant | Retriever returning noise / weak ranking |
| Context recall | Context | Retrieved chunks cover the info needed to answer | Retriever missing the right chunk entirely |
Real products, models, and research that use this idea.
- RAGAS is the open source standard for the four-metric split; teams import it to score faithfulness and context recall in CI on every RAG deploy.
- TruLens (TruEra) provides the 'RAG triad' (context relevance, groundedness, answer relevance) as feedback functions over a running pipeline.
What an interviewer would ask next. Try answering before peeking at the approach.
QFaithfulness is high but answer relevance is low. What's happening and how do you fix it?
The model grounds a confident answer to the wrong question or pads with on-context but off-query detail. Look at query rewriting, prompt instructions, and whether retrieval surfaced the right sub-topic at all.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Confusing context precision with context recall, or treating faithfulness and answer relevance as the same thing. They measure orthogonal failures and a pipeline can pass one while failing the other.
60 second bullets to scan on the way to the call.
The 2x2 split: context axis vs answer axis
Context precision vs context recall, and which one is the pipeline ceiling
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.