What are the four core RAGAS metrics and what does each measure?
RAGAS scores a RAG pipeline on four axes: faithfulness and answer relevance for generation, context precision and context recall for retrieval. The two generation metrics need no reference answer.
Imagine grading a student's open-book essay. You check four things. Did every claim in the essay actually come from the books they were allowed to use, or did they make stuff up (faithfulness)? Did the essay actually answer the question asked, or wander off (answer relevance)? Of the books they pulled off the shelf, how many were actually useful versus random clutter (context precision)? And did they pull every book they needed, or miss one that held a key fact (context recall)? The clever part is that the first two checks need no teacher's model answer. The usefulness check usually uses one, though it can run without it. The last check, recall, always needs a model answer, because you cannot know what was missed without knowing what should have been there.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
RAGAS (Retrieval Augmented Generation Assessment) is a landmark framework for evaluating RAG pipelines without large human-annotation budgets. The paper (Es et al., 2023) proposed three reference-free metrics: faithfulness, answer relevance and context relevance. They score a pipeline using only the question, the retrieved context, and the generated answer. The RAGAS library later added context precision and context recall. Recall always needs a human gold answer, and precision uses one by default.
A RAG pipeline has two components that can each fail independently. The retriever pulls context chunks from a knowledge base; the generator writes an answer conditioned on those chunks. A single holistic 'is this answer good' score cannot tell you which half broke. RAGAS solves this by assigning two metrics to the generator and two to the retriever, so the score pattern localizes the failure.
The four metrics and the two-component split
The RAGAS library's four core metrics split cleanly across the two halves of a RAG pipeline. The generation side gets faithfulness and answer relevance. The retrieval side gets context precision and context recall.
Faithfulness asks whether the answer's claims are grounded in the retrieved context, catching hallucination. Answer relevance asks whether the answer actually addresses the user's question, catching off-topic or evasive responses. On the retrieval side, context precision asks what fraction of the retrieved chunks are relevant to the question, penalizing noisy retrieval that buries signal in clutter. Context recall asks whether the retrieved context contains everything needed to answer fully, penalizing retrieval that misses a key chunk.
The diagnostic power is in the split. If faithfulness is low, the generator is inventing claims. If context recall is low, the retriever failed to fetch necessary evidence. If context precision is low, the retriever is fetching junk. Reading the four together tells you which component to fix rather than just that something is wrong.
This matters because before RAGAS the dominant RAG eval was either a single human rating per answer or a holistic LLM-judge score. Both collapse a two-stage system into one number, so a regression in next week's results gives no signal about whether to retrain the retriever or rewrite the generation prompt. The four-metric decomposition turns a vague quality complaint into a routed bug report, which is why the framework spread so fast through production stacks.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- RAGAS integrates with LlamaIndex and LangChain pipelines, exposing faithfulness and context-recall scorers.
- Teams run RAGAS faithfulness and answer relevance on live traffic with Claude Opus 4.7 or GPT-5.5 as the judge because those two are reference-free.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does context recall always require a reference answer?
Recall measures coverage of the information needed to answer. You cannot quantify what retrieval missed without a ground-truth answer that enumerates the required claims. Faithfulness and answer relevance need only the question, context, and generated answer, and context precision has a reference-free variant.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating all four RAGAS metrics as reference-free. Context recall always needs a ground-truth answer, and context precision uses one by default. Only faithfulness and answer relevance never need a human label.
60 second bullets to scan on the way to the call.
The four metric names and which component each one scores
Why faithfulness and answer relevance are reference-free, and when context precision is
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.