Streamingllm

Pair KV-cache eviction and KV quantization with the situation each one is the right answer for

Which token most often serves as an attention sink in pretrained autoregressive LLMs?

Why not just retrain models to eliminate attention sinks entirely?

Name two production KV-cache eviction policies and what each prioritizes

Find the bug: a Llama deployment serves traffic without prepending the BOS token.

Name two recovery moves a serving stack can pull when KV memory runs short mid request.

Spot the errors in this KV eviction strategy for long running generation

Why does naive sliding window KV eviction break and how does StreamingLLM's attention sink fix it?