Multiple Choice

In LoRA, where in the forward pass does `lora_dropout` apply its mask?

FP4 vs NF4, pick the answer that captures the actual structural difference

Which of these is actually saved when a fine-tuning checkpoint is written?

What does bnb_4bit_compute_dtype actually control in a QLoRA config?

bf16 vs fp16: which one preserves fp32's exponent range?

Base model vs instruction-tuned model: what does the user actually feel?

Where does cross-attention live in OpenAI Whisper?

At which step in the attention pipeline does the mask actually take effect?

Why split V from K instead of just reusing K as the value vector?

Tying K and V to one projection, what is gained and what is risked?

Dividing pre-softmax attention scores by an extra factor > 1 at inference does what?

Pick what happens to attention weights when pre-softmax scores are divided by a temperature T > 1

Along which axis of the QK^T score matrix is softmax applied inside attention?

Which token most often serves as an attention sink in pretrained autoregressive LLMs?

Llama-3 bumps RoPE base from 10k to 500k, which long-context property follows?

RMSNorm versus LayerNorm, what is kept and what is dropped?

Contrast absolute and relative positional encoding by what the score depends on

During decode, why is only Q computed for the new token while full K and V come from cache?

Pre-norm versus post-norm: which placement makes deep stacks stable?

vLLM defaults to a page size of 16. Pick what makes 1 and 1024 worse choices.

DeepSeek trains models to predict K tokens per step. Pick how that stays causal.

True or false: swapping a single-head attention layer for a 12-head one (same d_model) raises the parameter count.

Identify the attention-pattern difference between Mistral 7B and Llama-2 7B

INT8 KV cache vs INT8 weight quantization, which one is easier in production?

Head-pruning studies report that many attention heads can be removed with negligible loss. Pick the correct interpretation.