You are the tech lead on a fresh LLM-powered product. A skeptical senior asks whether picking up LangChain (or LlamaIndex, or Mastra) actually pays for itself versus calling the provider SDK directly. Give the conditions under which a framework earns its keep, the concrete taxes it imposes, and the decision rule you would apply.
Free · 2 AI evals / day
A framework like LangChain, LlamaIndex, or Mastra is not free. It saves you time on some things and costs you time on others. The skill is naming which is which.
**What a framework saves you.** Composition (chaining a prompt to a retriever to a parser to a tool to another prompt); provider abstraction (one interface that swaps between OpenAI, Anthropic, and others); observability hooks (callbacks that the major tracing vendors implement natively); and higher-order patterns (agent loops, streaming with tools, structured-output retry).
**What a framework costs you.** A lag of weeks to months behind the newest provider features (prompt caching, computer use, prediction outputs); types that bleed into your application code and are hard to remove later; an observability choice that is sometimes coupled to your framework choice (LangSmith only sees LangChain); and a real day in the future when you may want to remove the framework, which is more work than removing a thin wrapper would have been.
The decision rule is to match the framework to the shape of the work. One or two prompts: skip the framework. Three or more chains, an agent, or a routed RAG pipeline: take the framework hit early.
**Conditions a framework earns its keep:**
1. **Composition complexity.** When the app is multi-step chains, agents, RAG with re-ranking, parallel calls, or streaming with tools, hand-rolling the orchestration becomes its own codebase. LCEL, LlamaIndex's QueryEngines, and Mastra's workflows absorb that work.
2. **Multi-provider portability.** When the product needs to A/B between OpenAI and Anthropic, or to fail over from one provider to another, the framework's unified interface saves a rewrite. The cost is paid at install, not at code time.
3. **Native observability.** When the chosen tracing vendor (LangSmith, Langfuse, Arize Phoenix, Helicone) ships a first-class integration through the framework's callback surface, you get spans for free; without the framework you wire OpenTelemetry by hand.
**Taxes the framework charges:**
- **Feature lag.** OpenAI ships prompt caching on Tuesday; the framework wrapper catches up in 4 to 8 weeks. If you need the feature on day one, you write a temporary raw-SDK bypass anyway.
- **Type coupling.** Framework types (Runnable, ChatPromptValue, AgentAction) bleed into your application signatures. Removing the framework later is not a one-line change.
- **Observability lock-in.** Picking LangSmith pins you to LangChain. Picking Langfuse keeps the OTel exit open. The vendor choice is partly a framework choice.
- **Ejection cost.** The day the framework's abstractions stop matching your needs, removing it is real work. Bounded, but real.
**Decision rule.** If your surface is one or two prompts you can paste in a screen, stay on the raw SDK. If you are building three or more chains, agents, or routed pipelines, take the framework hit early.
Quick reference
TL;DR
A framework earns its keep when composition, multi-provider portability, or vendor-native observability matter; it taxes you on feature lag, type coupling, lock-in, and ejection cost.
Picture buying a fully outfitted RV versus buying a pickup truck and bolting on whatever you need. The RV is faster to drive away with on day one (kitchen, bed, lights, all wired up), but you pay for the parts you do not use, you wait for the manufacturer to ship the next upgrade, and selling it later is a big project. The pickup is slower on day one but flexible forever. The right pick depends on whether you are camping for a weekend (pickup wins) or living on the road for a year (RV pays off). LLM frameworks are the RV; the raw provider SDK is the pickup.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Single-prompt surfaces where composition is essentially zero The framework's value is composition; without composition, you pay all the tax for none of the win. The raw SDK is faster to write and easier to debug.
Hard latency budgets where framework dispatch overhead is measurable Even a few milliseconds of framework overhead per call can blow a budget on a hot path. The raw SDK is the conservative pick when every millisecond counts.
Day-one need for a provider feature the framework has not yet wrapped If the wrapper lags 4-8 weeks behind the provider, you will write the raw-SDK bypass anyway. Better to start there and avoid the partial commitment.
Quick scan · 2–4 min
Everything important, quickly.
7 min: four jobs frameworks do + three conditions for earning keep + four taxes imposed + decision rule + lifecycle skew + three production patterns + ejection mechanics.
Real-world examples
Real products, models, and research that use this idea.
1Klarna's customer-support stack started on LangChain, migrated critical paths to thin SDK wrappers as the surface stabilised; representative of the prototype to production pattern.
2Cursor reportedly runs raw provider SDK calls for the latency-critical autocomplete path, framework code for the broader chat surface.
3Linear's AI features use LangGraph for the multi-step workflows and raw SDK for the single-prompt features, the pragmatic 'right tool per surface' approach.
4Vercel's own AI SDK is the closest to a thin by design framework; teams that pick it explicitly are choosing low coupling.
5Mastra's pitch in 2026 is exactly the eval-first observability angle, demonstrating that framework choice and observability choice continue to converge.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWalk through a concrete ejection story.
Sign in or go Pro to see the approach for this follow-up.
QHow does the observability vendor decision actually couple to the framework decision?
Sign in or go Pro to see the approach for this follow-up.
QWhen does a framework that is NOT LangChain (DSPy, BAML, Mastra) make more sense?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Picking a framework as a matter of taste ('I like LangChain') instead of as a function of how much composition complexity the project will accumulate. The framework should earn its keep against the work, not the engineer's preference.