Drag each answer to line up with its matching prompt
RAG specific metrics, faithfulness, answer relevance, context precision, context recall, computed against a labeled or synthetic eval set
Ragas
User composed feedback functions (LLM-as-judge, classifier, custom) layered over traces of an LLM app
DSPy
Tracing first observability with evals as a layer over the stored trace store; Arize style analytics
TruLens
Compile loop where the optimizer evaluates candidate programs on a metric across a trainset. Eval is built into optimisation
Phoenix
Ragas = RAG metrics; TruLens = feedback functions; Phoenix = trace-first observability; DSPy = optimiser with eval in the compile loop.
Picture a restaurant kitchen with four different inspectors. Ragas tastes the food and grades each dish on specific qualities. Saltiness, doneness, presentation. TruLens watches the cook and asks custom questions like 'did they wash their hands' or 'did the dish match the order'. You write the questions. Phoenix is the security camera in the corner; it records everything that happens and lets you replay any night, then run inspections on the footage later. DSPy is the head chef who tastes a hundred versions of the same dish during a practice session and picks the recipe that scored best. Four different ways of asking 'is this LLM app any good', each born from a different starting concern.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Eval-integrated frameworks all look alike from a feature-list distance. They all compute scores over LLM outputs, most of them support LLM-as-judge, and several can produce a faithfulness number for a RAG app. From that distance, picking one looks like a taste call.
That distance is wrong. Each of these tools was built around a different primary abstraction, Ragas around a metric, TruLens around a feedback function, Phoenix around a span, DSPy around a program, and that abstraction shapes which workflows feel natural and which feel like fighting the framework. This dive walks through each, names the workflow it owns, and shows how to compose them without duplication.
Ragas. The RAG metric library
Ragas exists because the RAG community needed defensible, agreed-on metrics. Its four canonical scores became those metrics:
- Faithfulness. Does the answer stay grounded in the retrieved context?
- Answer relevance. Does the answer address the question that was asked?
- Context precision. Of what was retrieved, how much is actually relevant?
- Context recall. Of what is relevant, how much was retrieved?
Each is a function over a (question, retrieved_contexts, answer, ground_truth?) tuple, and most are computed by an LLM judge with a structured rubric. Ragas also generates synthetic test sets and supports a small but growing extension set (noise sensitivity, answer correctness).
The shape of a Ragas workflow: assemble an eval set, run your RAG pipeline over it, hand the resulting tuples plus the chosen metrics to evaluate(...), get a per-question and aggregated score back. The library makes no claim about tracing your app or improving it. It scores. That focus is its strength when you need numbers fast, and its limit when you need bespoke evaluators.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Framework | Primary primitive | Best for | Weakest at |
|---|---|---|---|
| Ragas | Canonical RAG metric | Defensible RAG scores fast | Custom non-RAG evaluators |
| TruLens | Composable feedback function | Bespoke product-specific evals | Out of box RAG metrics |
| Phoenix | OpenInference span / trace store | Trace exploration, drift analytics | Optimisation, prompt tuning |
| DSPy | Program + metric in a compile loop | Auto-tuning prompts and demonstrations | Production monitoring of a deployed app |
Real products, models, and research that use this idea.
- Ragas ships the canonical faithfulness, answer-relevance, context-precision, context-recall metrics used in most 2026 RAG evaluation papers.
- TruLens' feedback-function pattern powers Snowflake's internal LLM-as-judge eval pipelines after the TruEra acquisition.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you combine Phoenix and Ragas in one stack?
Phoenix captures traces from the live app; Ragas computes the canonical RAG metrics on the eval set or on sampled production traces. The two are layered, not competing. Phoenix is the store, Ragas is the evaluator.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating Ragas, TruLens, Phoenix, and DSPy as interchangeable 'eval tools' and picking by familiarity instead of by the workflow each one was designed for.
60 second bullets to scan on the way to the call.
Ragas' canonical four RAG metrics by name
TruLens feedback-function pattern
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.