What four metrics does the RAGAS paper propose and what does each measure?
RAGAS scores a RAG pipeline on four axes: faithfulness and answer relevance for generation, context precision and context recall for retrieval. Three are reference-free.
Imagine grading a student's open-book essay. You check four things. Did every claim in the essay actually come from the books they were allowed to use, or did they make stuff up (faithfulness)? Did the essay actually answer the question asked, or wander off (answer relevance)? Of the books they pulled off the shelf, how many were actually useful versus random clutter (context precision)? And did they pull every book they needed, or miss one that held a key fact (context recall)? The clever part is that for three of these you do not even need a teacher's model answer. You only need a model answer to check the last one, recall, because you cannot know what was missed without knowing what should have been there.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
RAGAS (Retrieval Augmented Generation Assessment) is a landmark framework for evaluating RAG pipelines without large human-annotation budgets. Its central contribution is a set of four metrics that are mostly reference-free: they score a pipeline using only the question, the retrieved context, and the generated answer, with a human gold answer required for just one of the four.
A RAG pipeline has two components that can each fail independently. The retriever pulls context chunks from a knowledge base; the generator writes an answer conditioned on those chunks. A single holistic 'is this answer good' score cannot tell you which half broke. RAGAS solves this by assigning two metrics to the generator and two to the retriever, so the score pattern localizes the failure. The deep dive walks each metric, how it is computed, the reference-free property, and the limitations a senior engineer must keep in mind.
The four metrics and the two-component split
RAGAS proposes four metrics, split cleanly across the two halves of a RAG pipeline. The generation side gets faithfulness and answer relevance. The retrieval side gets context precision and context recall.
Faithfulness asks whether the answer's claims are grounded in the retrieved context, catching hallucination. Answer relevance asks whether the answer actually addresses the user's question, catching off-topic or evasive responses. On the retrieval side, context precision asks what fraction of the retrieved chunks are relevant to the question, penalizing noisy retrieval that buries signal in clutter. Context recall asks whether the retrieved context contains everything needed to answer fully, penalizing retrieval that misses a key chunk.
The diagnostic power is in the split. If faithfulness is low, the generator is inventing claims. If context recall is low, the retriever failed to fetch necessary evidence. If context precision is low, the retriever is fetching junk. Reading the four together tells you which component to fix rather than just that something is wrong.
This matters because before RAGAS the dominant RAG eval was either a single human rating per answer or a holistic LLM-judge score. Both collapse a two-stage system into one number, so a regression in next-week's results gives no signal about whether to retrain the retriever or rewrite the generation prompt. The four-metric decomposition turns a vague quality complaint into a routed bug report, which is why the framework spread so fast through production stacks.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- RAGAS is the default RAG-eval library on LlamaIndex and LangChain pipelines, exposing faithfulness and context-recall scorers out of the box.
- Teams run RAGAS faithfulness and answer relevance on live traffic with Claude Opus 4.7 or GPT-5.5 as the judge because those two are reference-free.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy is context recall the only RAGAS metric that requires a reference answer?
Recall measures coverage of the information needed to answer. You cannot quantify what retrieval missed without a ground-truth answer that enumerates the required claims. The other three need only the question, context, and generated answer.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating all four RAGAS metrics as reference-free. Only context recall needs a ground-truth answer; the other three score the pipeline without any human label.
60 second bullets to scan on the way to the call.
The four metric names and which component each one scores
Why faithfulness, answer relevance, and context precision are reference-free
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.