Transformer Architecture

38 questions

End-to-end design of the transformer block: embeddings, attention, FFN, residual, normalization, positional encoding. Encoder vs decoder vs encoder-decoder.

Sort

Questions

Name a 2026 task where an encoder-decoder transformer still beats a decoder-only LLM
Short AnswerEasy·Qual 4.0
What fraction of a 7B transformer lives in its embedding table at vocab 128k?
Multiple ChoiceMedium·Qual 4.0
Identify the locations where dropout typically sits inside a transformer block
Multi-selectMedium·Qual 4.0
Spend a fixed parameter budget: more layers or wider d_model, and what does the evidence say?
Short AnswerHard·Qual 4.0
Why did decoder-only architectures take over tasks T5-style encoder-decoders used to win?
Short AnswerMedium·Qual 4.0
Complete the pre-norm transformer block recipe
Fill in BlankEasy·Qual 4.0Mistral AI
Order the operations inside a single pre-norm transformer block forward pass
Order StepsMedium·Qual 4.0Mistral AI
Predict the typical layer count for a Llama 7B and a Llama 70B
Predict OutputMedium·Qual 4.0
Pair each 2017 transformer ingredient with its 2026 production replacement
Match PairsHard·Qual 4.0
Which of these are linear layers that 'all-linear' LoRA targets in a SwiGLU transformer?
Multi-selectMedium·Qual 4.0
Explain why transformers replaced RNNs for language modeling.
Short AnswerEasy·Qual 4.6
Which components make up a standard transformer block?
Multiple ChoiceEasy·Qual 4.5Mistral AI
Explain scaled dot-product attention.
Short AnswerMedium·Qual 4.8

Showing 26–38 of 38

← PrevNext →