Anthropic

Define tied embeddings and explain why Llama-3 chose to untie them

Defend untying the output head: what does an extra ~500M parameters at the tail of the stack buy?

Which design choices from the original 'Attention Is All You Need' are no longer standard?

Same block, two regimes: training parallelism versus inference autoregression

Chinchilla's lesson: how did it reshape architecture and training choices after 2022?

Why does a deep transformer stack stop training if you remove the residual connections?

Describe the residual stream as a read/write surface that sublayers act on

Walk through why pre-norm trains stably past 100 layers while post-norm needs warmup tricks

Predict the trainable parameter count for a Llama-style 7B decoder block by block

Pick the recipe a 2026 frontier open-weight LLM block actually ships