ALiBi

Contrast absolute and relative positional encoding by what the score depends on

Name the additive bias term in attention and the two things it most commonly carries.

Contrast ALiBi's linear distance bias with RoPE's vector rotation, which one modifies scores additively, and which one modifies Q and K directly?

ALiBi, decode the acronym and pinpoint where in the pipeline it injects position

How does ALiBi inject position information without using positional embeddings?