Search

PremiumWhich part of a…

Walk through how a token's d_model embedding is split into per head pieces inside multi-head attention.

Identify the context vector inside scaled dot product attention and what produces it.

Contrast ALiBi's linear distance bias with RoPE's vector rotation, which one modifies scores additively, and which one modifies Q and K directly?

Dividing pre-softmax attention scores by an extra factor > 1 at inference does what?

Identify the attention pattern difference between Mistral 7B and Llama-2 7B

Why is causal attention cheaper than full bidirectional at the same length?

Pre-norm versus post-norm: which placement makes deep stacks stable?

Which sublayer of a dense transformer block carries more parameters, attention or FFN?

Interpret what the logit lens reveals when you unembed intermediate layers

PremiumExplain how a layer…

Walk through a T5 decoder block in order, list the attention sub-layers and what each one's Q, K, V read from.

Order the pipeline that stretches an 8k pretrained model to 128k context

PremiumOrder the operations inside…

Spot the masking bug in this packed sequence training setup

Critique this claim about CLIP using cross-attention between towers

PremiumWhy did decoder-only architectures…

PremiumExplain effective context for…

PremiumWhy does MQA underperform…

How does a Vision Transformer turn an image into a sequence, and what mask runs over it?

PremiumWhy does normalizing Q…

Define T5's relative position bias and what bucketing buys you

PremiumWhen do you need…

Chinchilla's lesson: how did it reshape architecture and training choices after 2022?

Describe the W_O projection in multi-head attention, its shape and what it mixes.

Why Hugging Face Hub functions as the de facto model registry for open weights

Name the scalar that QK^T is divided by inside scaled dot product attention.

Fill the blank: for a single head, QK^T has shape (T_query, ___).

Match GPT-2 versus Llama-2 attention design choices to their differences

PremiumMatch each positional encoding…