Defend the choice between LangSmith and Langfuse for an LLM heavy startup that may add Mastra and DSPy alongside LangChain in year two

Short answer·Hard·4.0 · 0·~3 min·Asked atCitadel Synthesia Writer

Attempt it

An LLM heavy startup uses LangChain today but expects to add Mastra (TS service) and DSPy (compile pipeline) in year two. They are choosing between LangSmith and Langfuse for observability. Make a recommendation and defend it on architecture grounds, not personal preference.

Free · 2 AI evals / day

Both LangSmith and Langfuse are observability platforms for LLM applications. They look similar in screenshots. Trace views, span trees, evaluation dashboards. The architectural difference is what they read from. LangSmith is built by the LangChain team and consumes LangChain's native callback events. It is the most polished experience if your stack is purely LangChain because it knows the framework's structure intimately. Langfuse is built on OpenTelemetry's GenAI semantic conventions. Any framework that emits OTel-compliant spans, LangChain via an instrumentation library, Mastra natively, DSPy via its tracer, feeds into Langfuse with the same shape. For the startup in the question, year two adds Mastra and DSPy. LangSmith does not have a path to ingest non-LangChain traces. Langfuse does. Both Mastra and DSPy can emit OTel spans that Langfuse accepts alongside the LangChain traces. The right call is Langfuse. The lock-in cost of LangSmith grows in year two, and the time to switch is harder once dashboards and eval datasets live there.

Pick Langfuse, on architecture grounds. The deciding fact is the framework heterogeneity in year two. LangSmith's data model is shaped around LangChain's callback events; the integration assumes the framework is LangChain. Mastra is a TypeScript framework with its own observability hooks, not LangChain callbacks. DSPy is a Python compile and execute system, also not LangChain. Routing those two through LangSmith would require parallel instrumentation pipelines, separate dashboards, and a fragmented eval story. Langfuse uses OpenTelemetry's GenAI semantic conventions as the ingest format. Instrument once against the conventions and all three frameworks flow into one trace store with comparable attributes. The team writes one observability stack and one query language, not three. Self-hosting matters too. Langfuse can run inside the team's VPC, which keeps prompts and PII off a third-party vendor. For a startup that may sell into regulated industries (healthcare, finance, EU sovereignty requirements) this is the difference between a six-month security review and a same-day adoption. LangSmith's legitimate strengths are real. The dataset-eval UI tied to LangChain runs is genuinely polished, and for a pure-LangChain shop it removes friction. But year-two the startup is not a pure-LangChain shop. The lock-in cost compounds as more dashboards and eval sets live in the proprietary backend. Making the portable choice early is asymmetrically cheap. Switching is hardest in year three, easiest in year one.

TL;DR

Pick Langfuse. It is OpenTelemetry-based, so LangChain, Mastra, and DSPy all flow into one trace store; LangSmith is LangChain-coupled and would leave two of three year-two frameworks dark.

Memory aid

Easy to grasp

Imagine choosing between two delivery companies for a growing business. Company A only delivers parcels from one specific shipping label brand. Company B accepts parcels from any brand that follows the standard postal format. Today you only ship with that one brand, so both companies work fine. Next year you plan to add two more brands. Pick company A and you will need separate delivery contracts for the new brands, or those parcels will pile up in your warehouse with nobody to pick them up. Pick company B and the new brands slot in for free because they all follow the same postal format. The right pick gets easier the further out you look.

Key concepts

LLM Frameworks LLM evaluation and observability

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

The LangSmith versus Langfuse choice is one of those decisions where the screenshot comparison hides the actual call. Both UIs look like trace explorers with span trees and evaluation views. The architectural difference is what they ingest, and that difference is what determines whether the choice ages well or becomes a future migration project.

This deep dive walks through the ingest formats, the four facts that decide the call for a multi-framework future, the honest cases where LangSmith remains the right pick, and the pragmatic adoption plan that follows from the recommendation.

The architectural divide: callbacks versus OpenTelemetry

LangSmith and Langfuse look superficially similar. Both render LLM traces, both support evaluation datasets, both have alerting and dashboards. The difference is one layer down, at the ingest format.

LangSmith ingests LangChain callbacks

LangSmith is built by the LangChain team. Its data model maps directly onto the callback events LangChain emits: on_llm_start, on_chain_start, on_tool_start, the run-id tree. The integration is one import and one environment variable, and every nuance of the LangChain runtime, chain composition, agent steps, retriever calls, shows up in the trace with the right shape.

Non-LangChain frameworks do not emit these callbacks. Mastra has its own observability surface. DSPy has its own tracer. Vercel AI SDK emits OpenTelemetry spans. None of them produce LangChain callback events, and LangSmith has no native way to ingest from them.

Langfuse ingests OpenTelemetry GenAI spans

Langfuse is built on the OpenTelemetry GenAI semantic conventions, a vendor-neutral specification for how LLM operations should be represented in traces. Attribute names like gen_ai.system, gen_ai.request.model, gen_ai.usage.input_tokens are standardized.

Any framework that emits OTel-compliant spans flows into Langfuse with the same shape. LangChain ships an OTel exporter that turns callbacks into OTel spans. Mastra ships native OTel support. DSPy can be instrumented with a generic OTel exporter around its tracer. The Langfuse instance ingests all three and renders them in a unified trace view.

What this means for the choice

The ingest format is the contract between the application and the observability backend. LangSmith's contract is a framework-internal API; Langfuse's contract is an industry-wide specification. As the application grows beyond one framework, the contract is what determines whether the backend keeps up.

The four facts that decide the call

Where LangSmith genuinely earns its keep

The pragmatic adoption plan

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Axis	LangSmith	Langfuse
Ingest format	LangChain callbacks	OpenTelemetry GenAI conventions
Multi-framework support	Requires parallel instrumentation	Native via OTel
Hosting	Vendor-hosted only	Vendor-hosted or self-hosted
Data residency	Vendor controls	Team controls
Eval dataset UI	Polished, LangChain-tight	Solid, framework-agnostic
Lock-in trajectory	Grows with usage	Portable via OTel

Real products, models, and research that use this idea.

Anthropic and OpenAI both publish OpenTelemetry GenAI traces from their server SDKs in 2026; both flow into Langfuse with no custom code.
Vercel ships AI SDK telemetry as OTel spans; the same Langfuse instance ingests them alongside LangChain.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you instrument a Mastra service so it emits compatible traces alongside the LangChain ones?

Use Mastra's built-in OTel exporter targeting the same Langfuse OTLP endpoint; ensure trace context propagation across service boundaries via the standard W3C TraceContext headers.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Picking LangSmith for the dataset-eval UI without thinking through what happens when the next service is not LangChain-shaped.

60 second bullets to scan on the way to the call.

The ingest format each platform uses
Why OpenTelemetry GenAI conventions matter for multi-framework stacks

Primary sources. Browse if you want the original framing.