Interpretability

Head-pruning studies report that many attention heads can be removed with negligible loss. Pick the correct interpretation.

What interpretability and downstream task concerns arise from allowing cross-word boundary merges in BoundlessBPE?

Why does multi-head attention outperform a single large attention head?

What does multi-head attention give you that a single attention head with the same total parameters cannot?

What has interpretability research shown about individual attention heads in trained transformers?