Causal Mask

At which step in the attention pipeline does the mask actually take effect?

A batch packs a causal LM with right padded sequences and applies only the causal mask. Spot the mistake.

What does a padding mask zero out, and why is it needed when batching variable length sequences?

Spot the masking bug in this packed sequence training setup

DeepSeek trains models to predict K tokens per step. Pick how that stays causal.

Why is causal attention cheaper than full bidirectional at the same length?

Encoder self-attention versus decoder self-attention, what is the single structural difference at the mask level?

Match each transformer family to the kind of attention it uses

For a causal mask over a 4-token sequence, what value sits at row 2, column 3?

Name the additive bias term in attention and the two things it most commonly carries.

PremiumSpot the error: 'I…

Predict the softmax output for a causal masked attention row.

PremiumWhy is a causal…