Zenaique
Topics
Practice
Browse
Reference
Challenges
Pricing
Search…
⌘K
Topics
/
LLM System Design
/
Capacity
Capacity
Subtopic
3 questions
Questions tagged with Capacity — part of LLM System Design.
Format
Difficulty
Role
Experience
Sort
Newest
Quality
Difficulty ↑
Difficulty ↓
Questions
Why capacity-plan an LLM service in tokens per second, not requests per second
Short Answer
Medium
·
Qual 4.0
A GPU pool sustains 9,000 output tokens/sec — will it hold this traffic?
Predict Output
Hard
·
Qual 4.0
Throughput, for an LLM serving system: give the working definition
Flashcard
Easy
·
Qual 4.0