When does the 'no framework' pattern, raw provider SDK plus your own small composition layer, beat reaching for LangChain or LlamaIndex? Give the concrete conditions, not generalities.
Free · 2 AI evals / day
The right way to think about framework choice is as a tax versus benefit tradeoff. A framework absorbs composition complexity, observability wiring, provider abstraction, and higher-order patterns like agent loops. In exchange you pay overhead. Extra dependencies, learning curve, type coupling, per-call latency cost, feature lag versus the underlying provider SDK.
The no-framework pattern, raw provider SDK plus your own small composition layer, wins when the things the framework absorbs are not actually present in your codebase.
Four concrete conditions tip the scale. First, small surface area: one or two prompts that you can hold in your head. There is no composition complexity for a framework to absorb. Second, day-one provider features: when you need OpenAI's brand-new capability this week, not in 4 to 8 weeks when the wrapper exposes it. Third, a one or two engineer team: you do not need a shared mental model across many contributors. Fourth, a tight latency budget: per-call overhead from callback dispatch and object construction matters at hundreds of QPS.
The framework wins at the other end of the spectrum. Complex composition, multi-engineer teams, mature observability requirements, agent loops with many tools. The decision is not ideological; it is whether the tax is justified for your specific workload.
Frameworks earn their keep by absorbing four things: composition complexity, provider abstraction, observability wiring, and higher-order patterns like agent loops. The no-framework pattern wins when these absorptions are not actually present in your codebase.
Four concrete conditions decide the call.
**Small surface area.** Two or three prompts in total, no chain composition, no agent loop. There is nothing for the framework to absorb. A 60-line file using `openai.AsyncOpenAI` directly is clearer than 60 lines of `ChatPromptTemplate` plus `RunnableSequence` plus `StrOutputParser` for the same result.
**Day-one provider features.** Provider SDKs ship new capabilities, OpenAI prompt caching, Anthropic computer-use, Gemini structured outputs, weeks or months before any framework wrapper catches up. If you need the feature this week, raw SDK is the only option.
**Small team.** One or two engineers do not need a shared mental model imposed by a framework's conventions. They can hold the system in their heads. The framework tax is largest when the team is smallest, because the maintainer count is zero and you pay all the cost yourself.
**Tight latency budget.** Per-call framework overhead, callback dispatch, object construction, type coercion, adds milliseconds. At hundreds of QPS with sub-second SLOs, that overhead is the difference between hitting the SLO and missing it.
The framework wins at the opposite end: multi-prompt systems with complex composition, many engineers needing shared conventions, mature observability requirements, agent loops with many tools, RAG pipelines with rerankers and routers.
The failure mode to avoid is reflexive adoption. Reaching for LangChain because everyone else does, on a project where the framework absorbs almost nothing, is paying tax for no return.
Quick reference
TL;DR
Raw SDK beats a framework when the surface is small, the team is tiny, the latency budget is tight, or you need a brand-new provider feature before the wrapper catches up.
Picture a single person making a sandwich for themselves. They open the bread bag, slap on cheese, done. Now picture them buying a 12-step sandwich-assembly machine for the same task. The machine has a process, settings, error logging, and a manual. For one sandwich a day it is silly. The setup time alone is longer than the task. The machine starts to earn its keep when you are making three hundred sandwiches with twelve different fillings for a catering company. That is when the structure stops being overhead and starts being help.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
A complex agent loop with many tools, conditional routing, and HITL pauses LangGraph or LlamaIndex Workflows absorbs real complexity that raw SDK would reinvent badly
A multi-step RAG pipeline with rerankers, hybrid retrievers, and routers LlamaIndex's named-stage pipeline is the right level of structure; raw SDK ends up recreating it
A team of 15+ engineers contributing to LLM code across many services Shared conventions imposed by a framework pay off at scale; without them, every team reinvents the wheel
Quick scan · 2–4 min
Everything important, quickly.
5 minutes: the tax vs benefit framing, the four conditions, what the thin-wrapper pattern looks like, and where frameworks still win.
Real-world examples
Real products, models, and research that use this idea.
1Many YC startups in 2024-26 publicly described their first-version stacks as raw OpenAI SDK plus a 50-line internal helper.
2Vercel AI SDK is itself a deliberate thin-wrapper choice. Small surface, type-safe, eject-friendly.
3Stripe and Shopify engineering posts describe internal LLM packages closer to wrappers than to frameworks.
4Cursor and Replit Agent expose feature-rich provider integrations because they need day-one access to new capabilities.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you decide when an internal LLM helper has grown enough that it should become a framework?
Sign in or go Pro to see the approach for this follow-up.
QWhat does a 50-line internal LLM helper actually contain?
Sign in or go Pro to see the approach for this follow-up.
QHow do you measure the per-call latency overhead of a framework versus raw SDK?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Reaching for LangChain by reflex on a project with two prompts and one engineer, then paying the framework tax for years on something that never needed it.