RAG-specific metrics, faithfulness, answer-relevance, context-precision, context-recall, computed against a labeled or synthetic eval set
User-composed feedback functions (LLM-as-judge, classifier, custom) layered over traces of an LLM app
Tracing-first observability with evals as a layer over the stored trace store; Arize-style analytics
Compile loop where the optimizer evaluates candidate programs on a metric across a trainset. Eval is built into optimisation
These four tools all touch evaluation, but each one started from a different problem.
Ragas began as a library of RAG-specific metrics. The big four, faithfulness, answer-relevance, context-precision, context-recall, are its signature. You run them over a labeled or synthetic eval set and get scores per question.
TruLens started from the idea of feedback functions. You compose your own evaluators, LLM-as-judge, classifiers, custom Python, and layer them over traces of your app. The library is about the composition, not a fixed metric set.
Phoenix from Arize is a tracing platform first. It captures the spans your app emits and gives you a UI to explore them; evals are a layer that runs over the stored traces.
DSPy is an optimiser. Its compile loop tries many candidate programs (prompt variants, demonstrations) against a metric on a trainset and keeps the best. Eval is not a separate step. It is built into how DSPy improves the program.
**Birth story decides the match.**
Ragas was born to answer 'how do I score this RAG app on the metrics RAG people argue about'. Its core surface is the four canonical RAG metrics, faithfulness, answer-relevance, context-precision, context-recall, plus extensions like noise-sensitivity. Bring an eval set, get scores per question, ship.
TruLens was born to answer 'how do I write my own evaluator and run it over my app's traces'. The primitive is the *feedback function*. A callable that returns a score for some piece of a trace. You compose them; the library gives you the plumbing, the LLM-as-judge helpers, and the dashboard. No fixed metric set is assumed.
Phoenix was born to answer 'where do my traces live and how do I explore them'. It is a tracing platform with an analytics UI; evals are a *layer over the trace store* you can run on stored data, not the starting concern.
DSPy was born to answer 'why am I writing prompts by hand'. The compile loop generates candidate programs and evaluates them against a metric across a trainset, keeping the best. Eval is not an output. It is the optimisation signal that drives the loop.
The match falls out: metrics → Ragas, custom feedback → TruLens, trace store → Phoenix, optimiser → DSPy.
Quick reference
TL;DR
Ragas = RAG metrics; TruLens = feedback functions; Phoenix = trace-first observability; DSPy = optimiser with eval in the compile loop.
Picture a restaurant kitchen with four different inspectors. Ragas tastes the food and grades each dish on specific qualities. Saltiness, doneness, presentation. TruLens watches the cook and asks custom questions like 'did they wash their hands' or 'did the dish match the order'. You write the questions.
Phoenix is the security camera in the corner; it records everything that happens and lets you replay any night, then run inspections on the footage later. DSPy is the head chef who tastes a hundred versions of the same dish during a practice session and picks the recipe that scored best.
Four different ways of asking 'is this LLM app any good', each born from a different starting concern.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Non-RAG LLM app being evaluated with Ragas Its metric set is RAG-shaped; for an agent or a classifier you are forcing a frame and getting noisy scores.
Small team wanting one canonical RAG score with no eval infra to build TruLens makes you write feedback functions before you have results; Ragas gets you to a number in one afternoon.
Team needing prompt-level optimisation in production hot path DSPy's compile loop is offline by design; running it per request would blow latency and cost budgets.
Team that already runs Datadog or Honeycomb with OTel and only needs eval scores Phoenix duplicates the tracing layer; Ragas or TruLens plugged into existing traces is the lighter integration.
Quick scan · 2–4 min
Everything important, quickly.
5 min: each framework's birth story, primary primitive, sweet spot, and how to combine them without overlap.
Side-by-side
Framework
Primary primitive
Best for
Weakest at
Ragas
Canonical RAG metric
Defensible RAG scores fast
Custom non-RAG evaluators
TruLens
Composable feedback function
Bespoke product-specific evals
Out of box RAG metrics
Phoenix
OpenInference span / trace store
Trace exploration, drift analytics
Optimisation, prompt tuning
DSPy
Program + metric in a compile loop
Auto-tuning prompts and demonstrations
Production monitoring of a deployed app
Real-world examples
Real products, models, and research that use this idea.
1Ragas ships the canonical faithfulness, answer-relevance, context-precision, context-recall metrics used in most 2026 RAG evaluation papers.
2TruLens' feedback-function pattern powers Snowflake's internal LLM-as-judge eval pipelines after the TruEra acquisition.
3Arize Phoenix is the open-source tracing layer many teams pair with LangChain or LlamaIndex for production span capture.
4DSPy's MIPROv2 and BootstrapFewShot optimisers are used in production at Databricks and at several research labs to tune RAG prompts against held-out trainsets.
5Langfuse is the missing fifth, open-source observability with OTel-native traces, often paired with Ragas as the eval layer.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you combine Phoenix and Ragas in one stack?
Sign in or go Pro to see the approach for this follow-up.
QWhen does DSPy's optimiser-driven eval fail to transfer to production?
Sign in or go Pro to see the approach for this follow-up.
QWhat does LangSmith do that none of these four does?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Treating Ragas, TruLens, Phoenix, and DSPy as interchangeable 'eval tools' and picking by familiarity instead of by the workflow each one was designed for.