Which statement correctly describes LangChain's `with_fallbacks` failure semantics?

MCQ·Medium·4.0 · 0·~1 min·Asked atAnthropic OpenAI Microsoft

Attempt it

**`with_fallbacks`** is LangChain's primitive for ordered failover. You wrap a Runnable with a list of backup Runnables; on an exception from the primary, the framework tries the first backup; if that also raises, it tries the second; and so on. If every backup fails, the original exception (or a chained exception) is re-raised. The shape is `primary.with_fallbacks([backup1, backup2, backup3])`. The primary runs first; backups run only if the prior call raised an exception. **What `with_fallbacks` does NOT do:** - **It does not retry the same Runnable.** That is `with_retry`, a different primitive with different semantics (and an exponential-backoff policy you can configure). - **It does not run concurrently.** Backups fire sequentially after the prior call fails; the call tree is linear. - **It does not silently swallow exceptions.** If all backups fail, the call raises; the application has to handle the final failure. The typical use case is provider failover: primary is `ChatOpenAI`, backup is `ChatAnthropic`. If OpenAI returns an error (rate limit, server error, model overloaded), the call automatically tries Claude with the same prompt. The user sees a slightly slower response but no failure.

**`with_fallbacks` is ordered failover semantics.** The wrapper holds an ordered list of backup Runnables. When the primary Runnable raises an exception (configurable via the `exceptions_to_handle` parameter, by default `Exception`), the wrapper invokes the first backup with the same input. If that backup also raises, the wrapper invokes the second backup. If every backup fails, the final exception (typically chained back to the primary's original exception) is re-raised. **The mechanics matter for production.** The call is sequential, not concurrent; total latency on a multi-backup failure is the sum of all attempted calls' latencies. The exception filter (`exceptions_to_handle`) lets you scope failover to specific exception classes; without scoping, even programming errors (TypeError, ValueError) trigger failover, which is usually wrong. **Why each wrong option is wrong.** - Retrying the same Runnable is `with_retry`, which takes a different policy shape (wait strategy, stop strategy, retry on exception filter). - Running concurrently would be a parallel-fanout pattern, sometimes called 'race' semantics; LangChain has no first-class primitive for this and you would build it from `RunnableParallel` plus race logic. - Silently swallowing exceptions is the failure mode `with_fallbacks` carefully does NOT have; if all backups fail, the call raises so the application knows something is wrong. **The production interaction with `with_retry`.** Mature reliability stacks compose them: `primary.with_retry(stop_after_attempt(3)).with_fallbacks([backup.with_retry(stop_after_attempt(2))])`. Retry the primary a few times for transient errors; on persistent failure, failover to the backup; retry the backup too; only raise if everything fails. The composition is non-obvious but the right pattern for serious production use.

Quick reference

TL;DR

`with_fallbacks` is ordered failover: on exception, advance to the next backup Runnable; never retry the same one, never run concurrently, never silently swallow.

Memory aid

Easy to grasp

Imagine you have three friends to ask for a ride home. You call the first; if she does not answer, you call the second; if he does not answer, you call the third. You do not redial the first friend over and over, you do not call all three at once, and if all three fail to answer you give up and figure out a Plan D. That is exactly how `with_fallbacks` works: a list of friends, called in order, with no redialing.

Key concepts

LLM frameworks

Detailed answer & concept explanation

~8 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

5 min: failover semantics in detail + contrast with with_retry + the exception filter trap + production composition pattern + observability on fallback rate + when to skip fallbacks.

Real-world examples

Real products, models, and research that use this idea.

Provider failover patterns: primary `ChatOpenAI` with `gpt-5.5`, fallback to `ChatAnthropic` with `Claude Opus 4.7`, the canonical use case.
Cost-tier fallback: premium model as primary, cheaper model as backup; if the premium is rate-limited, the cheaper model serves the request.
Region failover for compliance: EU-region OpenAI as primary, US-region as backup; in normal operation only the EU serves traffic.
Local-model backup: hosted provider as primary, local Ollama-served model as the last-resort backup for hard-availability requirements.
Klarna's customer-support stack reportedly uses provider-fallback patterns to maintain availability during OpenAI rate-limit windows.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWalk through the composition pattern for retry plus fallback in detail.

QHow would you build race-style concurrent fallback if you wanted it?

QWhat is the observability implication when fallbacks fire?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Confusing failover (next Runnable on failure) with retry (same Runnable again). `with_fallbacks` is the first; `with_retry` is the second. Mixing them up causes thundering retries on the wrong call.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.