Search

PremiumYour dense retriever performs…

What do you gain and lose by tying K and V to one projection?

PremiumWhat does 'position wise'…

What is the goal and risk of injecting noise into pre-softmax attention scores?

How does torch.nn.functional.scaled_dot_product_attention relate to FlashAttention?

Head pruning studies report that many attention heads can be removed with negligible loss. Which is the correct interpretation?

PremiumWhat does multi-head attention…

PremiumExplain how a layer…

Why does the transformer block use LayerNorm or RMSNorm instead of BatchNorm?

Compare ALiBi's linear distance bias with RoPE's vector rotation. Which one modifies scores additively, and which modifies Q and K directly?

Identify the context vector in scaled dot product attention and what produces it.

Walk through how a token's d_model embedding is split into per head pieces inside multi-head attention.

What does the router inside a Mixture-of-Experts block do, per token?

Define attention: what it does, and how it differs from convolution and recurrence.

Step back prompting in retrieval: what is it and when does it pay off?

Why Hugging Face Hub functions as the de facto model registry for open weights

PremiumWhy did the original…

PremiumWhy pre-norm trains stably…

PremiumWhy does MQA underperform…

PremiumExplain effective context for…

Walk through KV-cache updates when speculative decoding rejects some draft tokens

In a 7B transformer, do attention or MLP layers use more FLOPs, and does context length change that?

How does prefix KV-cache sharing reduce serving cost across requests?

Describe how continuous batching interacts with the KV cache

Name two production KV-cache eviction policies. What does each prioritize?

Multi-query versus decomposition: how do these query transforms raise recall on hard questions?

Spot the masking bug in this packed sequence training setup

Work out the KV cache reduction when 32 query heads share 8 KV heads

Calculate the per token KV cache footprint of a Llama-3-8B style config in fp16

Match GPT-2 versus Llama-2 attention design choices to their differences