Click any words you think contain an error. Click again to unmark.
Mark at least one word to submit.
The bug is the order of the two wrappers. Composition reads inside-out: the wrapper applied first is the innermost, and the wrapper applied last is the outermost. Exceptions bubble from inner to outer, so the inner wrapper sees them first.
As written, `primary.with_fallbacks([backup])` runs first, so the fallback is the inner wrapper. When OpenAI raises a transient error (429, timeout, server error), the fallback wrapper catches it immediately and switches to Anthropic. Anthropic responds successfully. The outer `.with_retry(stop_after_attempt=3)` never has a chance to retry the primary because the fallback already converted the failure into a success via the backup.
This defeats the intent. The author wanted to retry transient OpenAI failures up to three times before giving up and switching providers. What actually happens is a single OpenAI failure causes an immediate switch to Anthropic, and the retry only protects the composite. Which barely ever fails.
The fix is to swap the order. `primary.with_retry(stop_after_attempt=3)` first wraps the primary with retry. Now transient OpenAI errors get retried. Only after the retry budget is exhausted does the exception escape. The outer `.with_fallbacks([backup])` then catches it and switches to Anthropic.
The rule to remember: the wrapper you want to fire first goes innermost, which means it must be the first one applied.
The bug is wrapper-order reversal. LangChain composes Runnable wrappers inside-out. The first wrapper applied is the innermost; the last wrapper applied is the outermost. When the inner Runnable raises an exception, the innermost wrapper sees it first. Only if that wrapper re-raises does the next wrapper out get a chance.
The written order is `primary.with_fallbacks([backup]).with_retry(stop_after_attempt=3)`. Reading inside-out: primary is innermost, then fallback, then retry on the outside. When OpenAI raises a transient 429, the fallback (inner wrapper) catches it instantly and routes to Anthropic. Anthropic returns a response. The retry (outer wrapper) sees a successful call. There is nothing to retry. The author's intent to retry the primary three times before failing over never executes.
The corrected order is `primary.with_retry(stop_after_attempt=3).with_fallbacks([backup])`. Reading inside-out: primary innermost, retry wraps primary, fallback wraps the retry. On the same 429, the retry catches it and retries the primary up to three times. If all three retries fail, the exception escapes the retry boundary. The outer fallback catches it and switches to Anthropic.
The heuristic that prevents this mistake: write the wrappers in the order they should fire on a failure, where the first to fire is applied first. Or equivalently: the wrapper closest to the underlying call goes first; the wrapper that runs as the last line of defense goes last.
The same pattern matters for any composed Runnable wrappers. `with_listeners`, `with_config`, custom wrappers. Composition order is execution order in reverse.
Quick reference
TL;DR
The order is reversed: retry must be the inner wrapper and fallback the outer; as written, fallback fires on the first failure and the retry only protects the composite, which already succeeded via Anthropic.
Picture two safety nets stacked under a trapeze. The bottom net is the backup catcher; the top net is a second-chance bounce for the first catcher. If you put the bottom net on top by mistake, the moment the first catcher misses, the trapeze artist falls straight into the backup catcher's arms. The bounce never gets a chance to help. The bounce only matters when it is closer to the artist than the backup is. Same with retry and fallback: retry only helps if it wraps the primary call directly, with the fallback sitting outside it as the last resort.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Workloads where any provider switch is unacceptable for compliance or cost reasons Fallback to a different provider may violate data-handling rules; retry-only without fallback is the right pattern
Tight latency budgets that cannot afford retry delay Three retries on a 429 with exponential backoff can add seconds; failover to a faster backup may be preferable to retry
When the underlying SDK already does sophisticated retry Stacking framework-level retry on SDK retry can produce surprising compound delays; pick one layer and disable the other
Quick scan · 2–4 min
Everything important, quickly.
6 minutes: the inside-out composition rule, the buggy trace, the corrected trace, the generalization to other wrapper pairs, and the production three-layer stack.
Code example
python
# Wrong: fallback fires on first failure; retry never engages
reliable = primary.with_fallbacks([backup]).with_retry(stop_after_attempt=3)
# Right: retry attempts the primary; fallback only fires after retries exhaust
reliable = primary.with_retry(stop_after_attempt=3).with_fallbacks([backup])
Real-world examples
Real products, models, and research that use this idea.
1LangChain's own reliability documentation walks through this exact wrapper-ordering pattern as the canonical example.
2Several teams that adopted LangChain 0.3's reliability primitives report this bug as a common code-review catch.
3OpenAI and Anthropic SDKs ship native retry; relying on the SDK for retry and using with_fallbacks at the LangChain layer is the alternative pattern.
4Resilience libraries like Polly (.NET) and Resilience4j (JVM) document the same ordering rule for retry-fallback composition.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you add exponential backoff to the retry layer in this composition?
Sign in or go Pro to see the approach for this follow-up.
QWhat goes wrong if you put with_circuit_breaker as the innermost wrapper?
Sign in or go Pro to see the approach for this follow-up.
QHow would you trace the wrapper composition at runtime to confirm the order is correct?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Writing the wrappers left to right in the order you say the intent in English, instead of inside-out in the order the wrapping actually applies.