Zenaique
Topics
Practice
Browse
Reference
Challenges
Pricing
Search…
⌘K
Topics
/
Attention Mechanism
/
Attention Sink
Attention Sink
Subtopic
6 questions
Questions tagged with Attention Sink — part of Attention Mechanism.
Format
Difficulty
Role
Experience
Companies
Sort
Newest
Quality
Difficulty ↑
Difficulty ↓
Questions
Which token most often serves as an attention sink in pretrained autoregressive LLMs?
Multiple Choice
Easy
·
Qual 4.0
Why not just retrain models to eliminate attention sinks entirely?
Short Answer
Medium
·
Qual 4.0
Name two production KV-cache eviction policies and what each prioritizes
Short Answer
Medium
·
Qual 4.0
Find the bug: a Llama deployment serves traffic without prepending the BOS token.
Spot the Error
Medium
·
Qual 4.0
Spot the errors in this KV-eviction strategy for long-running generation
Spot the Error
Hard
·
Qual 4.0
Why does naive sliding-window KV eviction break and how does StreamingLLM's attention-sink fix it?
Short Answer
Hard
·
Qual 4.0