**Shadow deployment** is a way to test a new model (the challenger) against the one already in production (the incumbent) using real user traffic. Every incoming request is sent to both models. The incumbent's answer is the only one returned to the user. The challenger's answer is captured and stored for evaluation later.
The defining property is zero user exposure. Users never see the challenger's output, so even if the challenger is buggy or worse, no real user is harmed. That is what makes option A correct.
The other options describe things shadow deployment does not do. It is not slower for the user, because the incumbent's response is served immediately and independently of the challenger. It does not use synthetic traffic, it mirrors the real production stream. And it does not auto-promote the challenger, promotion is always a separate human or policy decision.
The trade-off is cost and signal. You pay for two model calls on every request, roughly doubling inference cost. And because the challenger's output is never shown, you get quality and latency data but no real user feedback like clicks or thumbs-up.
**Shadow deployment** mirrors real production traffic to a challenger model while serving only the incumbent's response to users. That single property defines it: real traffic, zero user exposure.
The challenger generates a full response for every mirrored request. That response is logged, not served. You then evaluate it offline against the incumbent. You score latency, cost, format validity, refusal rate, and quality via an LLM judge or human review. The user-facing path is untouched, so a broken challenger harms nobody.
Contrast this with A/B testing and canary. Both A/B and canary actually serve the challenger to some users. That exposes those users to potentially worse output. The upside is they collect real user-feedback signals: clicks, thumbs, conversion, retention. Shadow cannot collect any of those, because nobody ever sees the challenger.
The two costs are real. First, you run inference twice on every shadowed request, so spend roughly doubles. Second, the missing user-feedback signal means shadow is a pre-flight check, not a final verdict. The standard pattern is to shadow first to de-risk obvious regressions, then graduate to a small canary or A/B slice to capture true user response.
Quick reference
TL;DR
Shadow deployment mirrors real traffic to the challenger but serves only the incumbent's response. Users see zero challenger output, so it gives real-traffic evaluation at zero user risk.
Imagine a trainee chef shadowing the head chef. Every order that comes in, both of them cook it. But only the head chef's plate goes out to the customer. The trainee's plate is set aside so the kitchen manager can taste it later and compare. The customer never eats anything risky, yet you still learn how the trainee performs on real orders, not on a practice menu. The catch: you are now cooking every dish twice, so you burn through twice the ingredients and gas. And because nobody ever eats the trainee's food, you never find out whether real customers would have liked it.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
You need to measure user-facing outcomes like click-through, conversion, or retention Shadow never serves the challenger, so no user-feedback signal can ever be collected. Use a canary or A/B test.
The challenger has side effects, writing to databases, sending emails, or calling external tools Mirrored execution can corrupt shared state or trigger duplicate actions. Stub side effects or skip shadow entirely.
Inference budget is tight and request volume is very high Running both models on every request roughly doubles spend, which can dominate at high queries per second.
You already have a representative offline eval set and just need a quick regression check Offline eval is cheaper and faster when the production distribution is well covered. Skip the live mirror.
Quick scan · 2–4 min
Everything important, quickly.
4 min: shadow's defining zero-exposure property, what real-traffic mirroring uniquely buys, the no user signal and double-cost limits, and the shadow then canary then A/B promotion chain.
Side-by-side
Property
Shadow
Canary
A/B test
Challenger served to users
Never
Small slice
Treatment split
User exposure risk
Zero
Limited blast radius
Full on treatment arm
Real-traffic distribution
Yes
Yes
Yes
User-feedback signal collected
No
Partial
Yes, the deciding metric
Inference cost
Doubled, runs both models
Near baseline
Near baseline
Typical role in rollout
First de-risk gate
Early limited rollout
Final promotion decision
Real-world examples
Real products, models, and research that use this idea.
1LangSmith and Langfuse support logging a challenger's response alongside the served incumbent for offline shadow comparison on production traffic.
2Anthropic and OpenAI internal eval stacks mirror live prompts to candidate model versions before any user-facing rollout.
3Netflix popularized shadow traffic (dark launch) for backend services, the same pattern now applied to LLM challenger models.
4Promptfoo and Braintrust let teams replay captured production traffic against a new prompt or model as an offline shadow eval.
5Feature-flag platforms like LaunchDarkly gate the shadow then canary then A/B progression for LLM model swaps in 2026 stacks.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you safely shadow a challenger that calls tools or writes to a database?
Sign in or go Pro to see the approach for this follow-up.
QWhat metrics can shadow deployment measure, and which require a canary instead?
Sign in or go Pro to see the approach for this follow-up.
QHow do you decide when a clean shadow run justifies moving to a canary?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Treating shadow deployment as a full substitute for A/B testing. Shadow never serves the challenger, so it can never collect real user feedback signals like clicks, thumbs, or retention.