Causal Mask

At which step in the attention pipeline does the mask actually take effect?

A batch packs a causal LM with right-padded sequences and applies only the causal mask. Spot the mistake.

What does a padding mask zero out, and why is it needed when batching variable-length sequences?

Spot the masking bug in this packed-sequence training setup

DeepSeek trains models to predict K tokens per step. Pick how that stays causal.

Why is causal attention cheaper than full bidirectional at the same length?

Encoder self-attention versus decoder self-attention, what is the single structural difference at the mask level?

Match each transformer family to the kind of attention it uses

For a causal mask over a 4-token sequence, what value sits at row 2, column 3?

Name the additive bias term in attention and the two things it most commonly carries.

Spot the error: 'I apply the causal mask by setting masked positions to 0 in the attention scores before softmax.'

Predict the softmax output for a causal masked attention row.

Why is a causal mask required for autoregressive language model training?