Fresher / New Grad

Name the scalar that QK^T is divided by inside scaled dot-product attention.

RoPE, spell out the acronym and name exactly which vectors it rotates

Inside a transformer, what is the residual stream and how does attention interact with it?

Name what wraps the attention sub-layer in every transformer block.

Fill the blank: for a single head, QK^T has shape (T_query, ___).

During decode, why is only Q computed for the new token while full K and V come from cache?

Prefill and decode: name the two phases of LLM inference and say which one is compute-bound

What does a padding mask zero out, and why is it needed when batching variable-length sequences?

Given Q of shape (B, n_heads, T, d_head), the per-head attention output before concatenation has shape ___.

Describe the W_O projection in multi-head attention, its shape and what it mixes.

Which config token names the count of parallel attention heads in a layer?

MQA, spell out the acronym and name exactly what gets shared across heads

Decode the acronym MHA in the transformer attention context.

Across which axes is the KV cache stored, per layer, per head, per batch?

Compare hidden_size, d_model, and embedding_dim, three names for the same thing or three different things?

Pick the formula that defines head_dim from hidden_size and num_heads.

GQA, decode the acronym and describe its KV sharing pattern

Does FlashAttention change the attention output relative to standard attention?

Encoder self-attention versus decoder self-attention, what is the single structural difference at the mask level?

Match each transformer family to the kind of attention it uses

Inside a T5-style encoder-decoder, which sub-layer reads the encoder outputs, and through which projection?

Walk through how a token's d_model embedding is split into per-head pieces inside multi-head attention.

What does the dot product q . k actually measure inside an attention layer?

Walk through a T5 decoder block in order, list the attention sub-layers and what each one's Q, K, V read from.

Identify the context vector inside scaled dot-product attention and what produces it.