Walk through the capacity plan for launching an AI feature to 200,000 users
Your team launches an AI writing assistant to an existing base of 200,000 users next month, served from a hosted provider API. Walk through the capacity plan you would present: the math chain from users down to provider limits, and the actions the numbers should trigger.
Capacity planning is one multiplication chain: users to daily actives to calls to peak RPS to tokens per minute, checked against both TPM and RPM quotas with 2x headroom and a staged rollout.
Imagine opening a restaurant in a town of 200,000 people. Not everyone eats out daily, so you first guess how many actually show up, then how many dishes each person orders. Customers also do not arrive evenly: the lunch rush packs the room, so you size the kitchen for the rush, not for the daily average. Your kitchen has a supplier who caps how many ingredients you can buy per hour, so you call weeks ahead to raise that cap. Finally, you hold a soft opening for a few tables before opening every seat. An AI launch works the same way: the funnel is your guest count, tokens are the ingredients, and the provider quota is the supplier cap.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Most AI feature launches do not fail on model quality. They fail at 12:40 pm on day three, when the lunchtime peak crosses a token quota nobody had converted from the user forecast. Capacity planning for an LLM feature is the discipline of connecting two worlds that use different units: product thinks in users and sessions, providers enforce requests per minute and tokens per minute. The plan is the bridge.
This question shows up in system design interviews because it has no single right number; what is being tested is whether you can build a chain of explicit, challengeable assumptions, convert it into the units that actually throttle, and attach actions to the result. A plan that says 31 RPS peak is a forecast. A plan that says 31 RPS peak, therefore we filed for a quota raise to 8M TPM three weeks out and we ramp 5, 25, 100, is a launch plan.
We will walk the chain end to end with concrete numbers, then cover the operational moves the numbers should trigger.
The funnel: from user base to calls per day
Begin with the only number you are given: 200,000 existing users. Everything else is an assumption you write down explicitly so it can be corrected later.
First multiplier: daily active share. An assistant embedded inside an existing product might see 15 to 25 percent of the base on a given day; take 20 percent and you have 40,000 daily actives. Second multiplier: how often an active user triggers the AI. A writing assistant suggests, rewrites, and expands; 4 assisted actions per active day is a defensible starting point. That gives 160,000 calls per day.
Notice what makes this a good interview answer: each multiplier is named, sourced from a comparable product or an internal beta, and isolated. When week one data shows 9 percent DAU instead of 20, you correct one cell and the whole plan recomputes. When usage doubles because the feature is better than expected, same thing.
Two refinements are worth raising unprompted. Retries are real traffic: a 5 percent retry rate inflates calls by 5 percent, and it inflates them most during incidents, exactly when quota is tightest. And background invocations can dwarf explicit ones: if suggestions fire automatically on idle pauses while the user types, the per-user multiplier might be 40 rather than 4. Ask which kind of feature this is before defending any number.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
USERS = 200_000
DAU_SHARE = 0.20 # measure for real in week one
ACTIONS_PER_ACTIVE = 4
ACTIVE_HOURS = 10
PEAK_FACTOR = 7 # worktime product: 5-10x mean
TOK_IN, TOK_OUT = 1500, 400
calls_per_day = USERS * DAU_SHARE * ACTIONS_PER_ACTIVE
mean_rps = calls_per_day / (ACTIVE_HOURS * 3600)
peak_rps = mean_rps * PEAK_FACTOR
peak_rpm = peak_rps * 60
peak_tpm = peak_rpm * (TOK_IN + TOK_OUT)
print(f"calls/day {calls_per_day:,.0f}")
print(f"peak RPS {peak_rps:.1f} | RPM {peak_rpm:,.0f} | TPM {peak_tpm/1e6:.2f}M")
# Check BOTH peak_rpm and peak_tpm against quota; hold 2x headroom.Real products, models, and research that use this idea.
- OpenAI, Anthropic, and Google all publish tiered TPM and RPM limits per organization; planning a GPT-5.5 or Claude Sonnet 4.6 launch starts from those tier tables.
- AWS Bedrock enforces model level quotas raised through service quota requests, a process measured in days, which is why the ask goes in weeks early.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would the plan change if the provider sells provisioned throughput instead of shared quota?
Commit a provisioned baseline for steady state and burst into shared quota for the peak; compare unit economics at your actual utilization, since provisioned capacity priced for the peak sits idle most of the day.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Planning on the daily average. A worktime product peaks at 5 to 10 times its mean rate, and provider quotas throttle at the peak minute, not the average day.
60 second bullets to scan on the way to the call.
What multiplication chain turns a user base into calls per day?
Why plan on the peak minute rather than the daily average, and what peak factor fits a worktime product?
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.