Residual Stream

Sliding window attention can't see beyond the window. Describe the trick that recovers older context.

Inside a transformer, what is the residual stream and how does attention interact with it?

Pre-norm versus post-norm: which placement makes deep stacks stable?

Describe the W_O projection in multi-head attention, its shape and what it mixes.

Beyond shape preservation, name two things W_O does after head concat

Compare hidden_size, d_model, and embedding_dim, three names for the same thing or three different things?