Meta

Which design choices from the original 'Attention Is All You Need' are no longer standard?

Complete the RMSNorm formula

Walk through why pre-norm trains stably past 100 layers while post-norm needs warmup tricks

Match each positional encoding scheme to where it enters the architecture

Predict the trainable parameter count for a Llama-style 7B decoder block by block

Pick the recipe a 2026 frontier open-weight LLM block actually ships

Defend the 4x FFN expansion ratio, then explain why SwiGLU breaks the simple rule

Why did decoder-only architectures take over tasks T5-style encoder-decoders used to win?

Predict the typical layer count for a Llama 7B and a Llama 70B

Pair each 2017 transformer ingredient with its 2026 production replacement