`with_fallbacks` is ordered failover: on exception, advance to the next backup Runnable; never retry the same one, never run concurrently, never silently swallow.
Imagine you have three friends to ask for a ride home. You call the first; if she does not answer, you call the second; if he does not answer, you call the third. You do not redial the first friend over and over, you do not call all three at once, and if all three fail to answer you give up and figure out a Plan D. That is exactly how `with_fallbacks` works: a list of friends, called in order, with no redialing.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Production LLM stacks live and die by their reliability primitives. Provider outages, rate limits, and transient network errors are routine in 2026; an application that lacks a failover story is an application that goes down whenever its primary provider has a bad afternoon. LangChain's two reliability primitives, with_fallbacks and with_retry, address two distinct failure modes, and the gap between them is exactly where production bugs hide.
This MCQ tests whether the candidate can distinguish failover from retry, recognise that fallback semantics are sequential rather than concurrent, and know that failed fallbacks raise rather than silently swallowing errors. Each wrong option corresponds to a misconception that produces a different class of production bug: 'retry semantics' is the most common slip, 'concurrent semantics' produces wrong cost expectations, and 'silent swallowing' produces hidden outages that nobody pages on.
This deep dive walks the exact failover semantics, the contrast with retry, the production composition pattern that combines both primitives, the exception-filter trap that defaults to handling too much, the observability signal that mature stacks watch, and the cases where with_fallbacks is the wrong tool entirely.
Exactly what `with_fallbacks` does
with_fallbacks wraps a Runnable with an ordered list of backup Runnables. The wrapped object is itself a Runnable; calling it invokes the primary first. If the primary raises an exception (filtered by exceptions_to_handle, defaulting to Exception), the wrapper invokes the first backup with the same input. If that backup also raises, the wrapper invokes the second backup. The pattern continues through the list. If every backup fails, the wrapper raises the final exception, typically chained back to the primary's original exception via Python's __cause__ mechanism.
The shape is primary.with_fallbacks([backup1, backup2, backup3]) with optional keyword arguments: exceptions_to_handle=(SomeException,) scopes which exceptions trigger failover; exception_key=None controls whether the original exception is passed to the backup; fallback_value=None is not a default value but a hook for advanced patterns.
The mental model. Think of with_fallbacks as 'try a different thing.' The primary is one Runnable; each backup is a different Runnable. They might call different models, different providers, different prompts, different chains entirely. The only constraint is that they accept the same input shape and produce the same output shape, so the calling code does not know which one served the request.
The call tree is linear. Primary, then (on failure) backup1, then (on failure) backup2. The total latency on a multi-backup failure is the sum of all attempted calls' latencies. A primary that times out at 30 seconds followed by a backup that takes 2 seconds produces a 32-second total. This is usually fine because the failure path is rare, but it matters for SLO calculations: the p99 latency of a with_fallbacks chain is the worst-case path through every backup, not the median path.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Provider failover patterns: primary `ChatOpenAI` with `gpt-5.5`, fallback to `ChatAnthropic` with `Claude Opus 4.7`, the canonical use case.
- Cost-tier fallback: premium model as primary, cheaper model as backup; if the premium is rate-limited, the cheaper model serves the request.
What an interviewer would ask next. Try answering before peeking at the approach.
QWalk through the composition pattern for retry plus fallback in detail.
primary = ChatOpenAI(...).with_retry(stop_after_attempt(3), wait_exponential_jitter(initial=1, max=10)). backup = ChatAnthropic(...).with_retry(stop_after_attempt(2)). chain = primary.with_fallbacks([backup], exceptions_to_handle=(APIError, RateLimitError)). Each Runnable retries its own transient errors; only if a Runnable exhausts its retries does the fallback chain advance.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Confusing failover (next Runnable on failure) with retry (same Runnable again). `with_fallbacks` is the first; `with_retry` is the second. Mixing them up causes thundering retries on the wrong call.
60 second bullets to scan on the way to the call.
What the with-fallbacks wrapper does (ordered failover, sequential, raises on all-fail)
What the with-retry wrapper does and how it differs from fallbacks
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.