Estimate the average and peak QPS behind a million support tickets a month
A support copilot drafts replies for human agents. It handles 1,296,000 tickets per month (treat a month as 30 days) and makes exactly 2 LLM calls per ticket. Traffic peaks at 10x the average during business hours. Estimate (a) the average LLM calls per second and (b) the peak QPS you should capacity plan for.
1,296,000 tickets times 2 calls is 2,592,000 calls; a 30 day month is 2,592,000 seconds, so the average is exactly 1 call per second, and the 10x peak factor makes 10 QPS the planning number.
A bakery brags that it sells two and a half million buns a month. Sounds enormous, until you notice a month has about two and a half million seconds. The bakery sells one bun per second. One oven could nearly keep up. But customers do not arrive evenly. At lunch rush, ten people line up every second, and at 3 a.m. nobody comes. If the bakery built its kitchen for the average, the lunch line would stretch around the block every single day. So the rule is: divide the big monthly number into a per-second number to see how small it really is, then size the kitchen for the rush, not for the quiet average.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Every LLM system design interview starts, or should start, with a number like this one: a million-something events a month, delivered with a straight face to see whether the candidate flinches. The entire skill is one division, yet it routinely changes the conversation, because most big monthly numbers shrink into modest per-second rates the moment you do it.
This walkthrough does the division carefully, explains why the average and the peak are different numbers with different jobs, and then follows the estimate to where it actually leads in an LLM system: tokens per second against a provider rate card.
From monthly volume to a per-second rate
The conversion has exactly three steps, and each step has one trap.
Step one: count events at the right granularity. The system handles 1,296,000 tickets, but each ticket fires 2 LLM calls, so the unit that hits the API is calls: 2,592,000 of them. The trap is dividing tickets directly, which silently halves your answer. Always ask what actually consumes capacity: calls, not tickets; tokens, not calls, when you go one level deeper.
Step two: count the seconds. A day is 86,400 seconds; a 30 day month is 30 x 86,400 = 2,592,000 seconds. That constant, roughly 2.6 million, is worth memorizing because it converts any monthly volume to a per-second rate in one move.
Step three: divide.
The numbers match exactly because the problem was built that way, and noticing the match quickly is part of the test. In the wild you get rates like 0.7 or 3.2 per second, and the right move is rounding to one significant figure: this is an estimate that informs architecture, not a billing calculation. One call per second is the honest size of a million-ticket month, and it is small.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenAI and Anthropic both publish rate limits in requests per minute and tokens per minute per tier, so converting peak QPS onward into tokens per minute is exactly how teams pick a tier.
- Azure OpenAI provisioned throughput units are sized against peak tokens per minute, making the peak conversion, not the monthly average, the purchasing decision.
What an interviewer would ask next. Try answering before peeking at the approach.
QExtend the estimate: each call averages 3,000 input and 500 output tokens. What token throughput must the provider sustain at peak?
Multiply peak QPS by per-call token counts to get 30,000 input and 5,000 output tokens per second, then compare against rate tier limits in tokens per minute.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Dividing tickets instead of calls, or capacity planning on the average. The system must survive business-hour peaks, so the 10x peak number is the one you provision for.
60 second bullets to scan on the way to the call.
How many seconds are in a day, and roughly in a 30 day month?
Why do you count LLM calls rather than tickets when estimating request rate?
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.