How does softmax turn an attention score into an attention weight?

Attempt it

An attention score is the pre-softmax raw dot product (after scaling by 1/sqrt(d_k)). Scores are any real number, often negative, and a single row of the score matrix does not sum to anything in particular. Applying softmax along the key axis exponentiates each entry and normalizes by the row sum. The result is an attention weight in [0, 1] with each row summing to exactly 1, a valid probability distribution over keys. Softmax is the single step that crosses the boundary between score and weight.

Your answer

Quick reference

TL;DR

Score is the pre-softmax raw dot product (any real number). Weight is the post-softmax probability in [0, 1] that rows sum to 1. Softmax is the bridge.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Think of judges at a talent show. First, each judge writes a raw score for every act on a piece of paper. The numbers can be anything: positive, negative, all over the place, and there is no rule that they have to add up to anything. Then the host runs through the room collecting those scores, runs them through a calculator, and converts them so each act lands as a clean percentage between 0% and 100%, with all the percentages across acts adding up to exactly 100%. The raw paper numbers are what the model calls 'scores'. The final percentages are what it calls 'weights'. The host with the calculator in between is the single step that converts one into the other.

Key concepts

Attention score Attention weight Softmax Probability simplex

Detailed answer & concept explanation

~5 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

Define score as the pre-softmax dot product and weight as the post-softmax probability, list which operations live at each level (masking, temperature, biases), and explain why the distinction matters for stable softmax and FlashAttention.

Real-world examples

Real products, models, and research that use this idea.

Vaswani et al. 2017: the canonical attention diagram explicitly separates 'MatMul + Scale' (score) from 'SoftMax' (weight) as distinct boxes.
Attention visualization tools (BertViz, exBERT) plot the weight matrix, not the score matrix; that is why the heatmap rows sum to 1.
Causal masking in every modern LLM (GPT-5.5, Llama 4 Maverick, Claude Opus 4.7, Gemini 3.1 Pro) is implemented at the score level by adding -inf to future positions before softmax.
ALiBi (BLOOM, MPT) adds a per-head linear bias to scores before softmax; doing it post-softmax would break the probability constraint.
FlashAttention v2 and v3 compute scores tile by tile, maintain running softmax statistics, and never materialize the full weight matrix in HBM.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy are causal masks applied at the score level rather than the weight level?

Sign in or go Pro to see the approach for this follow-up.

QHow does the row max trick for numerically stable softmax work, and does it change the weights?

Sign in or go Pro to see the approach for this follow-up.

QWhat does 'attention map' mean in a paper, and is it scores or weights?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Using 'score' and 'weight' interchangeably. They are different objects living in different spaces; softmax is the operator that converts one to the other.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.