Zenaique
Topics
Practice
Study
Browse
Reference
Pricing
Search…
⌘K
Topics
/
Inference Optimization
/
Bandwidth Bound
Bandwidth Bound
Subtopic
3 questions
Questions tagged with Bandwidth Bound — part of Inference Optimization.
Premium questions for this topic
Format
Difficulty
Role
Experience
Companies
Sort
Newest
Quality
Difficulty ↑
Difficulty ↓
Questions
Which quantization combo squeezes the most decode bandwidth per percent of quality lost?
Multiple Choice
Medium
Walk through the byte accounting that proves a single batch decode step is bandwidth bound on H100.
Short Answer
Hard
Predict the bandwidth vs compute latency of a single Llama-70B decode step on H100
Predict Output
Hard
Descript
NVIDIA
Intuit
NVIDIA
Airbnb
Graphcore