Zenaique
Topics
Practice
Browse
Reference
Challenges
Pricing
Search…
⌘K
Topics
/
Attention Mechanism
/
KV Cache Math
KV Cache Math
Subtopic
2 questions
Questions tagged with KV Cache Math — part of Attention Mechanism.
Format
Difficulty
Role
Experience
Companies
Sort
Newest
Quality
Difficulty ↑
Difficulty ↓
Questions
Why is KV cache memory the bottleneck for long context LLM serving?
Short Answer
Hard
·
Qual 4.0
Compute the KV cache memory for a single request at 4096 context on Llama-2 70B (MHA) in FP16.
Predict Output
Hard
·
Qual 4.0