List the selection criteria you would apply to a candidate LLM framework before adoption, and rank them by long-term consequence
Your team is evaluating a candidate LLM framework for a multi-year product. List the selection criteria you would run it through before adoption, and rank them by long-term consequence. Justify the ranking.
Rank by cost of being wrong: ejection path, observability, feature lag (architectural and irreversible) come before composition fit, language fit, streaming UX (tactical and adjustable).
Imagine choosing where to plant trees on a new property. Some choices are hard to undo. The root system grows wide, the trunk grows thick, and ten years in you cannot just move the tree. Other choices, like which annual flowers to plant, you can change every spring. Framing framework selection the same way: which decisions root deep and which can be redone next sprint. The ones that root deep need the most thought up front because the cost of being wrong is years of debt. Ejection cost is the deepest root. Streaming UX is the annual flower.
Detailed answer & concept explanation~6 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
6 min: the six criteria, the ranking principle, why ejection and observability beat composition fit, and how to defend the inversion in an interview.
| Criterion | Tier | Cost of being wrong | Mitigation |
|---|---|---|---|
| Ejection path | Architectural | Multi-quarter migration | Wrap behind own interfaces |
| Observability compatibility | Architectural | Vendor pinned for years | Pick OTel-conformant framework |
| Feature-lag tolerance | Architectural | Structurally behind the curve | Keep SDK-direct code paths |
| Composition complexity | Tactical | Dead weight or hand-rolling | Re-pick at sub-feature granularity |
| Team language fit | Tactical | Velocity tax | Pick TS-first for TS teams |
| Streaming UX | Tactical | Worse user experience | Switch streaming primitive per feature |
Real products, models, and research that use this idea.
- Notion's framework selection process documented their move to wrap LangChain behind internal interfaces to keep ejection cost bounded.
- Stripe engineering picked Langfuse over LangSmith partly because OTel-conformant traces kept the framework decision independent of the observability decision.
- Cursor and Replit ship SDK-direct paths for hot loops despite using frameworks elsewhere. Feature-lag tolerance drove the architectural split.
- Vercel AI SDK adoption in Next.js apps grew because TypeScript-first composition + streaming UX hit two top criteria for the user-facing chat segment.
- Mastra's eval-first framing in 2025-2026 targeted teams who picked observability and ejection criteria over composition richness.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you write the integration test that proves ejection cost is bounded?
QWhen would you reject a framework even if it ranked well on all six criteria?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Ranking by 'how often the criterion matters in daily work' (which puts composition fit at the top) instead of by 'how expensive the wrong decision becomes over time' (which puts ejection cost first).
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.