Match each LLM observability vendor to its 2026 positioning slogan
Drag each answer to line up with its matching prompt
Langfuse
Hosted, LangChain/LangGraph coupled, playground first iteration loop
LangSmith
Evaluation first product with tracing layered on
Arize Phoenix
Pure OTel instrumentation library, ships spans to any OTel backend
Helicone
HTTP proxy that logs every LLM call with one base URL change
Datadog LLM Observability
Open source, self-hostable, OTel native, prompt versioning and dataset evals
Braintrust
Open source, OpenInference native, RAG and embedding drift focused
OpenLLMetry (Traceloop)
Incumbent APM extended to LLMs, single pane of glass for orgs already on Datadog
Seven LLM-observability vendors, each with a distinct 2026 positioning: Langfuse (OSS OTel), LangSmith (LangChain coupling), Phoenix (RAG-focused OSS), Helicone (proxy), Datadog (APM extension), Braintrust
Think of a city's restaurant scene. Each one survives by being known for one specific thing. The neighborhood bakery is known for fresh bread. The steakhouse is known for steaks. The food court inside the big mall is known for being where everyone already goes for lunch. The home-delivery service is known for not requiring you to leave home. LLM observability vendors are the same: each picked a different lane. Langfuse picked open-source-plus-OpenTelemetry-friendly. LangSmith picked deep integration with one framework. Phoenix picked retrieval-quality visualizations. Helicone picked the easiest setup (change one URL). Datadog picked being the place where everything already lives. Braintrust picked evaluation as the front door. OpenLLMetry is not a place to eat; it is the standard-shaped takeout container that fits in any restaurant's kitchen.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
LLM observability went from a one-vendor category in 2023 (LangSmith) to a crowded field by 2026. Picking a vendor is now a real decision, and the structure of the field (who picked which lane) is the first thing to understand before comparing features.
This section maps each of the seven main vendors to its 2026 positioning, organizes them along three axes, and explains where OpenLLMetry fits as a non-backend instrumentation layer.
The seven vendors and their lanes
Langfuse. Open-source (MIT-licensed), self-hostable, OTel-native, with strong prompt-management and dataset-eval workflows. Available as Langfuse Cloud (hosted) and Langfuse Self-Hosted. Accepts OTLP/HTTP spans directly, so any OTel-instrumented app can target it. The default choice for teams that want OSS with a hosted upgrade path.
LangSmith. Hosted product from LangChain. Deeply coupled to LangChain and LangGraph. The playground-first iteration loop (edit prompt, see results, compare runs) is the canonical LangSmith workflow. The LangSmith Hub is its prompt-registry. Best fit when you are committed to the LangChain stack.
Arize Phoenix. Open-source, OpenInference-native, with strong RAG and embedding-drift visualizations. The OSS sibling of Arize AX (which adds production-scale storage, classical ML monitoring, and team features). Phoenix's retrieval-quality panels are best in class.
Helicone. HTTP proxy. You change your OpenAI base URL to Helicone's proxy URL; they log every request and response. One-line setup. Also offers an async-observe mode that decouples from the request path. The trade-off in proxy mode is they sit in the path: latency and reliability now depend on Helicone.
Datadog LLM Observability. Datadog's extension of its APM platform to LLM-specific signals. The pitch is single pane of glass: if you already run Datadog for APM, RUM, and logs, LLM traces land in the same UI with the same query language. The trade-off is that LLM-specific depth (RAG views, embedding drift) lags dedicated tools.
Braintrust. Evaluation-first product. Strong dataset management, judge frameworks (LLM-as-judge with built-in calibration), and experiment comparison views. Tracing is layered on but not the lead. Best fit when your primary workflow is offline and online evaluation, not real-time tracing.
OpenLLMetry (Traceloop). Not a backend. A pure OTel instrumentation library: import it, decorate your LLM calls, get gen_ai-shaped OTel spans for free. Ships those spans via OTLP to any OTel backend (Langfuse, Phoenix, Datadog, etc.). The emitter half of an OTel-native stack.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Vendor | Deployment | OTel-native? | Workflow lead |
|---|---|---|---|
| Langfuse | OSS + hosted | Yes | Tracing + prompts + datasets |
| LangSmith | Hosted | Limited | LangChain/LangGraph debugging |
| Arize Phoenix | OSS | Yes (OpenInference) | RAG + embedding drift |
| Arize AX | Hosted | Yes (OpenInference) | RAG + ML monitoring |
| Helicone | Hosted proxy | Limited | Setup simplicity |
| Datadog LLM Obs | Hosted (Datadog) | Yes | Single pane of glass |
| Braintrust | Hosted | Partial | Evaluation workflows |
| OpenLLMetry | Library (not a backend) | Yes | Instrumentation only |
Real products, models, and research that use this idea.
- OpenLLMetry from Traceloop is widely paired with Langfuse, Phoenix, and Datadog LLM Observability in production stacks; the same instrumentation feeds multiple backends.
- LangSmith's depth with LangGraph is the canonical example of acceptable vendor lock-in: teams on LangGraph rarely leave LangSmith because the integration value exceeds the portability cost.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you decide between Langfuse and Phoenix for a self-hosted OSS deployment?
Phoenix has stronger RAG and embedding-drift visualizations; Langfuse has stronger prompt-management and dataset workflows. Decide by which one's core feature set matches your dominant workload. Both are OTel-native, both are credibly production-quality.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Confusing OpenLLMetry (an instrumentation library) with the vendors that consume it; OpenLLMetry is the spans-emitter, not a backend.
60 second bullets to scan on the way to the call.
Each vendor's primary 2026 positioning slogan
The three axes (deployment, OTel-native, workflow priority) that organize the category
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.