Zenaique
Topics
Practice
Browse
Reference
Challenges
Pricing
Search…
⌘K
Topics
/
Inference Optimization
/
Throughput
Throughput
Subtopic
9 questions
Questions tagged with Throughput — part of Inference Optimization.
Format
Difficulty
Role
Experience
Companies
Sort
Newest
Quality
Difficulty ↑
Difficulty ↓
Questions
In benchmarks, tokens per second measures what exactly?
Flashcard
Easy
·
Qual 4.0
Throughput, for an LLM serving system: give the working definition
Flashcard
Easy
·
Qual 4.0
Sequence packing: what it does, and the attention-mask gotcha
Short Answer
Hard
·
Qual 4.0
Define TTFT, TPOT and throughput. Which should a chatbot vs a batch summarizer optimize?
Short Answer
Medium
·
Qual 4.0
A chatbot team reports TTFT P99 jumped from 300ms to 1.4s overnight. Which root cause is most likely?
Multiple Choice
Medium
·
Qual 4.0
What is the primary mechanism by which PagedAttention increases serving throughput?
Multiple Choice
Medium
·
Qual 4.0
Spot the errors in this description of continuous batching
Spot the Error
Medium
·
Qual 4.0
Why is batching the single biggest throughput lever for LLM decode?
Short Answer
Medium
·
Qual 4.0
What is the primary reason batching is so much more effective for LLM decode than for CNN inference?
Multiple Choice
Medium
·
Qual 4.0