Zenaique

Select every eval framework that is purpose built for RAG specific metrics

Multi-select·Medium·4.0 · 0·~1 min·Asked atIntuitLabelboxQualcomm
Attempt it
TL;DR

Ragas, TruLens, and DeepEval ship RAG-specific metrics out of the box (faithfulness, context precision, groundedness); Promptfoo and Inspect AI are general-purpose runners that need RAG metrics wired in.

Memory aid
Sign in to see the mnemonic that makes this stick.
Easy to grasp

Imagine buying a kitchen knife set. Some sets advertise themselves as 'fish knives' and come with the exact blades you need to fillet salmon. Others are 'general chef sets' that will technically cut fish but require you to bring your own filleting technique. Ragas, TruLens, and DeepEval are the fish-knife sets for RAG: they come with faithfulness, context precision, and groundedness measures already labeled and sharpened. Promptfoo and Inspect AI are excellent general sets that can do RAG work if you bring the metrics yourself, but they do not advertise RAG as the primary use case. When the question asks which are purpose-built for RAG, the answer is whichever ship the RAG-specific blades in the default box.

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

The LLM eval-framework landscape is crowded enough by 2026 that a typical candidate has used one or two and inferred the rest. The multi-select tests whether the candidate can correctly place five named frameworks on a single axis: does the framework ship RAG-specific metrics in its default metric library.

This explanation walks through what counts as a RAG-specific metric, places each of the five frameworks on that axis, and shows how mature stacks combine more than one in practice.

What counts as a RAG-specific metric

The canonical RAG quality framing has two halves: did we retrieve the right context, and did the answer use it correctly. Each half decomposes into a small set of measurable signals.

Retrieval-side metrics. Context precision: of the chunks the retriever returned, how many were actually relevant to the question. Context recall: of the chunks that would have been relevant, how many did the retriever return. Both depend on having either ground-truth relevance labels or an LLM-as-judge that can adjudicate relevance.

Generation-side metrics. Faithfulness (also called groundedness): of the claims in the answer, how many are supported by the retrieved context. Answer relevance: does the answer actually address the question that was asked, independent of whether the underlying facts are correct.

A framework that 'ships RAG metrics' is one where these four (or close variants) are first-class imports in the metric library, with default judge prompts and default thresholds. A framework that lets you build them yourself is general-purpose.

Placing the five frameworks
Why the question is sometimes mis-answered
How mature stacks compose more than one
Sign in to unlock the full deep dive.

Situations where this technique stops working.

Sign in to see when this approach fails.

2–4 min · Everything important, quickly.

Sign in to see the quick scan of the deep dive.

Real products, models, and research that use this idea.

  • Ragas inside a GitHub Action: on every PR to the retrieval prompt, the action runs faithfulness and context recall on the golden set and posts a delta comment.
  • TruLens wired into a LlamaIndex query engine to log groundedness on every dev-environment query, surfacing regressions before they hit CI.
Sign in to see more production examples.

What an interviewer would ask next. Try answering before peeking at the approach.

QCompare the faithfulness implementations across Ragas, TruLens, and DeepEval. Where do they actually differ?
A

All three use an LLM-as-judge. They differ in judge prompt design (statement-decomposition in Ragas, single-pass scoring in TruLens, claim extraction then verify in DeepEval), in how they handle multi-turn contexts, and in their default judge model choice. The behavior on edge cases (long context, partial groundedness, paraphrased citations) drifts apart between the three, which is why teams running mixed stacks usually pick one as the production gate and treat the others as cross-check.

1 more follow-up an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Picking only Ragas because the name says RAG. TruLens and DeepEval both ship dedicated RAG metric suites; the question tests whether the candidate knows the broader landscape, not just the most-named tool.

Sign in to see all red flags and common mistakes.

60 second bullets to scan on the way to the call.

  • The four canonical RAG metrics: faithfulness, answer relevance, context precision, context recall

  • Which three frameworks ship these out of the box

Sign in to unlock the revision sheet.

Primary sources. Browse if you want the original framing.

Similar questions

Same topic, related formats. Practice these next.

4 curated
Next question
Why a circuit breaker around the primary LLM provider is more than a fancy retry
Flashcard·Medium