True or false: switching from 1-head to 12-head attention at the same d_model increases total parameter count.

**Multi-head attention** is a structural reinterpretation of single head attention, not an additive change. A quick vocabulary anchor: `d_model` is the model's hidden dimension (768 for BERT-base). A **head** is one parallel attention computation. `d_head` is the per-head feature dimension, defined as: $$d_\text{head} = \frac{d_\text{model}}{n_\text{heads}}$$ In single head attention, the projection weights W_Q, W_K, W_V each have shape (d_model, d_model). The forward pass projects input x to q, k, v of dimension d_model, runs attention, then projects back with W_O of shape (d_model, d_model). Total parameters: 4 × d_model². In multi-head, the matrices W_Q, W_K, W_V still have shape (d_model, d_model). The forward pass projects to a d_model vector and then **reshapes** that dimension into (num_heads, d_head). Each head runs its own attention over its slice. The outputs are concatenated back to d_model and projected by W_O. Same 4 × d_model² parameters; reshape, not addition.

**False.** Multi-head attention reshapes the `d_model × d_model` Q/K/V matrices into num_heads parallel d_model × d_head blocks, where d_head = d_model / num_heads. Total parameter count and total FLOPs are unchanged. The way I'd think about it: the underlying matrix is the same size. You're just choosing whether to interpret the output dimension as one chunk of size d_model or as num_heads parallel chunks of size d_head. Storage is identical, and the matmul work decomposes into smaller per-head matmuls that sum to the same total FLOPs. What does change is structural: you get num_heads parallel attention distributions per query instead of one, which enables head specialization. That's the actual reason multi-head exists, same parameter budget, more expressive routing.