Zenaique
Topics
Practice
Study
Browse
Reference
Pricing
Search…
⌘K
Topics
/
Inference Optimization
/
Saturation
Saturation
Subtopic
2 questions
Questions tagged with Saturation — part of Inference Optimization.
Premium questions for this topic
Format
Difficulty
Role
Experience
Companies
Sort
Newest
Quality
Difficulty ↑
Difficulty ↓
Questions
As batch size grows, where does throughput stop increasing and per request latency start exploding?
Short Answer
Hard
Predict the throughput and per request latency behaviour as batch size grows past saturation
Predict Output
Hard
NVIDIA
Sigmoid
Intuit
NVIDIA