**`with_fallbacks`** is LangChain's primitive for ordered failover. You wrap a Runnable with a list of backup Runnables; on an exception from the primary, the framework tries the first backup; if that also raises, it tries the second; and so on. If every backup fails, the original exception (or a chained exception) is re-raised.
The shape is `primary.with_fallbacks([backup1, backup2, backup3])`. The primary runs first; backups run only if the prior call raised an exception.
**What `with_fallbacks` does NOT do:**
- **It does not retry the same Runnable.** That is `with_retry`, a different primitive with different semantics (and an exponential-backoff policy you can configure).
- **It does not run concurrently.** Backups fire sequentially after the prior call fails; the call tree is linear.
- **It does not silently swallow exceptions.** If all backups fail, the call raises; the application has to handle the final failure.
The typical use case is provider failover: primary is `ChatOpenAI`, backup is `ChatAnthropic`. If OpenAI returns an error (rate limit, server error, model overloaded), the call automatically tries Claude with the same prompt. The user sees a slightly slower response but no failure.
**`with_fallbacks` is ordered failover semantics.** The wrapper holds an ordered list of backup Runnables. When the primary Runnable raises an exception (configurable via the `exceptions_to_handle` parameter, by default `Exception`), the wrapper invokes the first backup with the same input. If that backup also raises, the wrapper invokes the second backup. If every backup fails, the final exception (typically chained back to the primary's original exception) is re-raised.
**The mechanics matter for production.** The call is sequential, not concurrent; total latency on a multi-backup failure is the sum of all attempted calls' latencies. The exception filter (`exceptions_to_handle`) lets you scope failover to specific exception classes; without scoping, even programming errors (TypeError, ValueError) trigger failover, which is usually wrong.
**Why each wrong option is wrong.**
- Retrying the same Runnable is `with_retry`, which takes a different policy shape (wait strategy, stop strategy, retry on exception filter).
- Running concurrently would be a parallel-fanout pattern, sometimes called 'race' semantics; LangChain has no first-class primitive for this and you would build it from `RunnableParallel` plus race logic.
- Silently swallowing exceptions is the failure mode `with_fallbacks` carefully does NOT have; if all backups fail, the call raises so the application knows something is wrong.
**The production interaction with `with_retry`.** Mature reliability stacks compose them: `primary.with_retry(stop_after_attempt(3)).with_fallbacks([backup.with_retry(stop_after_attempt(2))])`. Retry the primary a few times for transient errors; on persistent failure, failover to the backup; retry the backup too; only raise if everything fails. The composition is non-obvious but the right pattern for serious production use.
Quick reference
TL;DR
`with_fallbacks` is ordered failover: on exception, advance to the next backup Runnable; never retry the same one, never run concurrently, never silently swallow.
Imagine you have three friends to ask for a ride home. You call the first; if she does not answer, you call the second; if he does not answer, you call the third. You do not redial the first friend over and over, you do not call all three at once, and if all three fail to answer you give up and figure out a Plan D. That is exactly how `with_fallbacks` works: a list of friends, called in order, with no redialing.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Workflows where the backup's output would be semantically inconsistent with the primary's Failover assumes the backup is a drop-in replacement. If models have meaningfully different output styles (e.g. mixing a strict-format primary with a chatty backup), the user experience may degrade in unexpected ways.
Workflows where latency on the failure path matters more than availability Sequential failover sums the latencies. A primary that times out at 30s followed by a backup that takes 2s gives the user 32s of latency. If your SLO is tighter than this, race-style concurrency (built outside the with-fallbacks wrapper) may be the right pattern.
Workflows where the failure is your own bug, not a provider issue The wrapper routes programming errors to backups, masking the bug. Scope the exception filter to provider-related exceptions only.
Quick scan · 2–4 min
Everything important, quickly.
5 min: failover semantics in detail + contrast with with_retry + the exception filter trap + production composition pattern + observability on fallback rate + when to skip fallbacks.
Real-world examples
Real products, models, and research that use this idea.
1Provider failover patterns: primary `ChatOpenAI` with `gpt-5.5`, fallback to `ChatAnthropic` with `Claude Opus 4.7`, the canonical use case.
2Cost-tier fallback: premium model as primary, cheaper model as backup; if the premium is rate-limited, the cheaper model serves the request.
3Region failover for compliance: EU-region OpenAI as primary, US-region as backup; in normal operation only the EU serves traffic.
4Local-model backup: hosted provider as primary, local Ollama-served model as the last-resort backup for hard-availability requirements.
5Klarna's customer-support stack reportedly uses provider-fallback patterns to maintain availability during OpenAI rate-limit windows.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWalk through the composition pattern for retry plus fallback in detail.
Sign in or go Pro to see the approach for this follow-up.
QHow would you build race-style concurrent fallback if you wanted it?
Sign in or go Pro to see the approach for this follow-up.
QWhat is the observability implication when fallbacks fire?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Confusing failover (next Runnable on failure) with retry (same Runnable again). `with_fallbacks` is the first; `with_retry` is the second. Mixing them up causes thundering retries on the wrong call.