Design a sensible migration plan from a LangChain-heavy codebase to a thinner SDK-based one

Short answer·Hard·4.0 · 0·~3 min·Asked atAnthropic OpenAI Microsoft

Attempt it

Your codebase is heavily LangChain-shaped. Every prompt is a ChatPromptTemplate, every chain is an LCEL pipeline, observability is LangSmith. Leadership wants to move toward a thinner raw-SDK posture without freezing the product. Sketch a sensible migration plan.

Free · 2 AI evals / day

Migrating off LangChain is a phased project, not a single switch. The goal is not to remove LangChain entirely. It is to right-size it, keeping it where it earns its keep and removing it where the framework tax exceeds the benefit. The order matters. Observability portability has to come first. LangSmith only sees LangChain runs; if you move a chain off LangChain without OpenTelemetry already flowing, you lose visibility into that chain. Switch the tracing backend to Langfuse or Phoenix on OTel before any rewrite. The second step is picking targets. Hot paths win first. High-traffic chains with low composition complexity, simple prompt and parse work. These give the biggest cost and latency win per unit migration effort. Rewrite each hot path as raw provider SDK plus a thin internal helper covering retry, structured output, and tracing. The third step is keeping LangChain where it still earns its keep. Composition-heavy flows, agent loops with many tools, complex RAG with rerankers and routers, are exactly the workloads LangChain absorbs the most complexity for. Leave them on the framework. Measure progress by counting call sites that hold LangChain types in their signatures. That number is the type-coupling debt; it is the migration budget. A six to twelve month phased plan typically removes 80 percent of LangChain dependence with no product freeze.

The migration is staged, measurable, and selective. **Stage one: portability.** Switch observability from LangSmith to an OpenTelemetry-based backend (Langfuse or Phoenix) before touching any chain. LangSmith only ingests LangChain callbacks; any chain rewritten to raw SDK without OTel in place becomes invisible. Validate that OTel is flowing for current LangChain traffic before stage two. **Stage two: target selection.** Inventory the LangChain call sites. Sort by traffic and composition complexity. The right migration targets are high-traffic and low-complexity. Endpoints that do prompt and parse, simple structured-output, basic streaming. These have the highest win per unit effort because the framework is absorbing very little for them. **Stage three: hot-path rewrites.** For each chosen target, replace the LangChain chain with raw provider SDK (`openai`, `anthropic`, `google.genai`) plus a thin internal helper. The helper is 5 to 50 lines: retry with exponential backoff, structured-output parsing (Pydantic or Zod), and OTel span emission. Keep the helper signature stable so the call sites change minimally. **Stage four: hold the line.** Composition-heavy flows stay on LangChain. Agent loops with many tools, multi-step RAG with rerankers and routers, complex chain orchestration. These are exactly what the framework absorbs complexity for. Migrating them costs more than it saves. **Stage five: governance.** Set the rule that any new code path must justify its framework dependency. New chains do not inherit LangChain by default; they pick the lightest tool that solves the actual problem. **Measurement.** Track type-coupling debt: count the call sites that hold LangChain types (ChatPromptTemplate, RunnableSequence, BaseChatModel) in their signatures. That count is the migration budget. Burn it down over six to twelve months. A done-well migration typically removes 70 to 90 percent of LangChain dependence without freezing product development.

Quick reference

TL;DR

Switch observability to OpenTelemetry first, then migrate hot paths to raw SDK, keep LangChain for composition-heavy work, and measure type-coupling as the migration budget.

Memory aid

Easy to grasp

Picture moving out of a furnished apartment over many months instead of one weekend. You do not show up with a truck and rip out everything at once. You start with what is easiest to replace, like the dishes you use every day. You hold onto the heavy custom shelves until you have a place to put them. You also install your own utility meters before you move anything so you can prove the new place uses less electricity. The point is not to leave with zero items; it is to keep only what is still earning its rent.

Key concepts

LLM Frameworks LLM evaluation and observability

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

6 minutes: the five stages, the dependency chain that fixes the order, hot-path selection criteria, the thin-wrapper pattern, and the right-sizing framing.

Real-world examples

Real products, models, and research that use this idea.

Several Anthropic customer case studies in 2025-26 describe phased LangChain right-sizing rather than full removal.
Shopify's public engineering posts on LLM infrastructure describe thin-wrapper patterns over the provider SDK with OTel-based tracing.
Stripe's prompt-engineering platform reportedly uses raw SDK plus internal helpers for hot paths, LangGraph for orchestration where it earns its keep.
Klarna's customer-service stack migrated portions off LangChain to thin wrappers while keeping LangGraph for the agent loop.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you handle a chain that uses 12 different LangChain tools. Does it stay or go?

QWhat is the minimum eval coverage you would require before approving a hot-path rewrite?

QHow do you prevent regression when an engineer rebuilds a LangChain chain in the rewrite era?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating migration as rip and replace and trying to flag-day every chain off LangChain, which freezes the product without proportionate benefit.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.