Zenaique

Estimate the average and peak QPS behind a million support tickets a month

Predict output·Easy·4.0 · 0·~2 min·Asked atCitadelH2o AiLlamaIndex
Attempt it
A support copilot drafts replies for human agents. It handles 1,296,000 tickets per month (treat a month as 30 days) and makes exactly 2 LLM calls per ticket. Traffic peaks at 10x the average during business hours. Estimate (a) the average LLM calls per second and (b) the peak QPS you should capacity plan for.
TL;DR

1,296,000 tickets times 2 calls is 2,592,000 calls; a 30 day month is 2,592,000 seconds, so the average is exactly 1 call per second, and the 10x peak factor makes 10 QPS the planning number.

Memory aid
Sign in to see the mnemonic that makes this stick.
Easy to grasp

A bakery brags that it sells two and a half million buns a month. Sounds enormous, until you notice a month has about two and a half million seconds. The bakery sells one bun per second. One oven could nearly keep up. But customers do not arrive evenly. At lunch rush, ten people line up every second, and at 3 a.m. nobody comes. If the bakery built its kitchen for the average, the lunch line would stretch around the block every single day. So the rule is: divide the big monthly number into a per-second number to see how small it really is, then size the kitchen for the rush, not for the quiet average.

Key concepts

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Every LLM system design interview starts, or should start, with a number like this one: a million-something events a month, delivered with a straight face to see whether the candidate flinches. The entire skill is one division, yet it routinely changes the conversation, because most big monthly numbers shrink into modest per-second rates the moment you do it.

This walkthrough does the division carefully, explains why the average and the peak are different numbers with different jobs, and then follows the estimate to where it actually leads in an LLM system: tokens per second against a provider rate card.

From monthly volume to a per-second rate

The conversion has exactly three steps, and each step has one trap.

Step one: count events at the right granularity. The system handles 1,296,000 tickets, but each ticket fires 2 LLM calls, so the unit that hits the API is calls: 2,592,000 of them. The trap is dividing tickets directly, which silently halves your answer. Always ask what actually consumes capacity: calls, not tickets; tokens, not calls, when you go one level deeper.

Step two: count the seconds. A day is 86,400 seconds; a 30 day month is 30 x 86,400 = 2,592,000 seconds. That constant, roughly 2.6 million, is worth memorizing because it converts any monthly volume to a per-second rate in one move.

Step three: divide.

λavg=2,592,000 calls2,592,000 s=1 call/s\lambda_{avg} = \frac{2{,}592{,}000 \text{ calls}}{2{,}592{,}000 \text{ s}} = 1 \text{ call/s}

The numbers match exactly because the problem was built that way, and noticing the match quickly is part of the test. In the wild you get rates like 0.7 or 3.2 per second, and the right move is rounding to one significant figure: this is an estimate that informs architecture, not a billing calculation. One call per second is the honest size of a million-ticket month, and it is small.

Average versus peak: two numbers, two jobs
The next conversion: QPS to tokens per second
Estimation hygiene under interview pressure
Sign in to unlock the full deep dive.

Situations where this technique stops working.

Sign in to see when this approach fails.

2–4 min · Everything important, quickly.

Sign in to see the quick scan of the deep dive.

Real products, models, and research that use this idea.

  • OpenAI and Anthropic both publish rate limits in requests per minute and tokens per minute per tier, so converting peak QPS onward into tokens per minute is exactly how teams pick a tier.
  • Azure OpenAI provisioned throughput units are sized against peak tokens per minute, making the peak conversion, not the monthly average, the purchasing decision.
Sign in to see more production examples.

What an interviewer would ask next. Try answering before peeking at the approach.

QExtend the estimate: each call averages 3,000 input and 500 output tokens. What token throughput must the provider sustain at peak?
A

Multiply peak QPS by per-call token counts to get 30,000 input and 5,000 output tokens per second, then compare against rate tier limits in tokens per minute.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Dividing tickets instead of calls, or capacity planning on the average. The system must survive business-hour peaks, so the 10x peak number is the one you provision for.

Sign in to see all red flags and common mistakes.

60 second bullets to scan on the way to the call.

  • How many seconds are in a day, and roughly in a 30 day month?

  • Why do you count LLM calls rather than tickets when estimating request rate?

Sign in to unlock the revision sheet.

Primary sources. Browse if you want the original framing.

Similar questions

Same topic, related formats. Practice these next.

4 curated
Next question
In LLM serving, what is the primary driver of end to end latency for a generation request?
MCQ·Medium