q dot k is the unnormalized affinity score between a query and a key, combining direction (cosine of the angle) with vector magnitude.
Picture two people at a job fair pointing flashlights at each other. The query is your flashlight, the key is theirs. When you both shine straight at each other, the beams overlap a lot, strong match. When you face perpendicular directions, the beams barely cross, weak match.When you face away, the beams point apart, negative match. The dot product is just a number that says how much the two beams overlap. The model later turns that overlap into a probability and uses it to decide how much of the key's payload to pay attention to.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
The dot product is the workhorse of attention. Every transformer in production computes it billions of times per second, and yet it is one of the most under-explained components of the architecture. People can recite the softmax(Q K^T / sqrt(d_k)) V formula but freeze when asked what Q K^T actually means geometrically, why we scale by sqrt(d_k) specifically, or why we use dot product instead of cosine similarity in the first place.
This deep dive walks the geometry, the algebra, and the engineering, and ends with a precise statement of what the dot product gives you and what it does not.
Geometric meaning: alignment and magnitude
Pick any two vectors q and k in d_k-dimensional space. Their dot product is:
The second form is the geometric one. It splits the score into three intuitive pieces.
The cosine factor
cos(theta) is the alignment indicator:
theta = 0→ vectors point the same way → cos = 1 → maximum positive contribution.theta = 90 degrees→ orthogonal → cos = 0 → no contribution.theta = 180 degrees→ opposite → cos = -1 → maximum negative contribution.
The magnitude factors
|q| and |k| scale the entire result. Two perfectly-aligned unit vectors give a score of 1. Two perfectly-aligned vectors of length 5 each give 25. This is fundamentally different from cosine similarity, which strips the magnitude factor and returns only cos(theta).
Working takeaway: the dot product is alignment times scale. Aligned but small vectors lose to aligned and big vectors.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Every transformer-based LLM in production (GPT-5.5, Claude Opus 4.7, Llama 4 Maverick, Gemini 3.1 Pro, Qwen 3.5) computes attention via batched `Q K^T` dot products.
- FlashAttention-2 and FlashAttention-3 fuse the `Q K^T` dot product with the softmax and the value matmul to avoid materializing the full score matrix in HBM.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy exactly `sqrt(d_k)` for the scaling, not `d_k` or some other function?
Under unit-Gaussian q and k, the variance of q . k is d_k. Standard deviation is therefore sqrt(d_k). Dividing by sqrt(d_k) brings the score back to unit variance, which keeps softmax gradients from saturating. Dividing by d_k would over-shrink and waste signal range.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Conflating the raw dot product with cosine similarity. They share direction information, but the dot product also scales with vector magnitudes, that magnitude effect matters.
60 second bullets to scan on the way to the call.
State the geometric definition of the dot product in terms of magnitudes and angle
Three regimes (aligned, orthogonal, opposite) and their score signs
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.