Open-source Python library for RAG feedback functions (groundedness, context relevance, answer relevance). Niche: code-first, notebook-friendly eval without adopting a full observability platform.
Picture a baker who needs to test bread dough. They could install a giant industrial test rig in the kitchen, or they could carry a pocket thermometer and humidity stick that they pull out when they want. TruLens is the pocket toolkit for testing RAG outputs. It is a small library you import into your Python code, you call its 'is this answer grounded in the retrieved chunks' function on a single example, and you get a score. Other tools like Langfuse and Phoenix are the full industrial test rigs: more powerful and connected to a UI, but more to set up. TruLens stays useful when you want quick, code-only checks without standing up a platform.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
TruLens was an early mover in the RAG-evaluation space, shipping the feedback-function abstraction in 2023-2024 when most LLM observability platforms still focused on raw tracing. By 2026 the major platforms (Langfuse 3.x, Arize Phoenix 5.x, LangSmith) have absorbed similar evaluators as first-class features. So the natural question is: what niche does TruLens still occupy?
The honest answer is a narrower one than it occupied in 2023, but the niche is real and likely durable: code-first, library-level RAG evaluation for notebook and developer-flow use cases that do not need or want a full platform. This walkthrough explains what TruLens actually provides, why the library shape persists even after platforms grew similar evaluators, where TruLens stops being the right tool, and how to plan the migration if your needs outgrow it.
Mental model: TruLens is a Python library of RAG feedback functions. Platforms are full observability stacks. They overlap on evaluator content but differ on shape; the right choice depends on where the eval lives in your workflow.
What TruLens actually provides
The feedback-function abstraction
A feedback function is a callable that takes a structured input (typically the query, the retrieved context, the generated answer) and returns a score on a specific dimension.
from trulens.feedback import Groundedness
f_groundedness = Groundedness(provider=openai_provider)
score = f_groundedness.groundedness_measure_with_cot_reasons(
source=retrieved_context,
statement=generated_answer,
)
Under the hood, the feedback function is usually an LLM-as-judge prompt. You can swap the judge model, swap the prompt, or write entirely new feedback functions.
The canonical RAG triad
The three dimensions that anchor RAG quality:
- Context relevance: do the retrieved chunks match the query intent? Answers 'did retrieval work?'
- Groundedness: do the answer's claims trace back to the retrieved chunks? Answers 'did the model use retrieval honestly?'
- Answer relevance: does the answer address the user's question? Answers 'is the response useful?'
Tracking all three is more diagnostic than any one alone: high groundedness with low answer relevance means the model is citing dutifully but missing the point; high context relevance with low groundedness means retrieval works but the model is ignoring it.
Other feedback functions
TruLens ships toxicity, bias, sentiment, language match, and similar dimensions. They are less RAG-specific but useful when you need them.
Storage and visualization
TruLens has a local SQLite-backed dashboard (tru.run_dashboard()) for browsing evaluations. It is useful for notebook flows; it is not a production-grade observability UI.
Integration shape
You wrap your RAG chain in TruChain (or the framework-specific equivalent), declare which feedback functions to apply, and run. TruLens intercepts the calls, captures the inputs and outputs, and scores each call. The integration is library-level: it lives in your code, not in a separate service.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- TruLens is open source from TruEra (now part of Snowflake); the library remains actively maintained in 2026.
- Data science teams at companies prototyping RAG features use TruLens in notebooks for quick iteration without standing up platform tooling.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you migrate from TruLens feedback functions to Langfuse evaluators when you outgrow the library?
Extract the LLM-as-judge prompt text from the TruLens feedback function. Port it into a Langfuse evaluator with the same prompt, judge model, and rails. The prompt is the portable artifact; the surrounding orchestration is what changes.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating TruLens as competing with Langfuse on full-platform features. It is a library, not a platform; the comparison is unfair to both.
60 second bullets to scan on the way to the call.
TruLens as a Python library, not a platform
The feedback-function abstraction
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.