Long Context

PremiumExplain many-shot jailbreaking and…

For 1M-token context decode, which parallelism axis actually relieves KV pressure?

Name the dominant cost driver of GPT-5.5 latency at a 200k-token input

Dividing pre-softmax attention scores by an extra factor > 1 at inference does what?

PremiumExplain effective context for…

Which token most often serves as an attention sink in pretrained autoregressive LLMs?

PremiumOrder the three steps…

Llama-3 bumps RoPE base from 10k to 500k, which long context property follows?

Name two production KV-cache eviction policies and what each prioritizes

In a 7B transformer, do attention or MLP layers eat more FLOPs, and does context length change the answer?

How are RoPE NTK scaling and YaRN used at inference time to extend context beyond the training length?

How much does FP8 KV cache help, what does it cost and what is KIVI?

Compute the KV cache size for a 70B model at 128k context (80 layers, 64 heads, head_dim 128, FP16).

Where does FlashAttention deliver the biggest serving speedup?

PremiumWhat's the 'attention sink'…

Match each sparse attention pattern to its design and exemplar architecture.

What problem does ring attention solve that data parallelism cannot?

PremiumWhy is KV cache…

Match each long context strategy to what it modifies.