Standard pooling choices in 2026 are mean (encoder default), last-token (decoder-only embedders), and learned attention pool; CLS is fine for classification but not for sentence similarity.
Imagine a school class of 32 students and you need one report card that summarizes the class. Mean pool: average everyone's grades. Last-token pool: ask the last student, who has heard everyone else (only works in classes where students speak in order). Attention pool: ask a teacher to weight each student by importance. CLS pool: ask the kid who sits in seat 1 (they were not specifically asked to summarize, so the answer is patchy). Sum without normalization: add the grades but do not divide, so a class of 40 looks twice as smart as a class of 20. Random pick: pick a random student. Only the first three give sensible report cards.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Pooling choice is one of the few architectural decisions in embedding models that has a clear right answer per model family. Mean pool for encoders, last-token pool for decoder-only embedders, learned attention pool for the small set of models that squeeze out the last percent. CLS, sum without norm, and random pooling are all wrong, but they are wrong for different reasons, which is what makes this a good multi-select.
This deep dive covers the three standards by their architectural justification, the three rejects by their specific failure modes, and the production gotchas that come up when porting embedding code between model families.
Mental model: the pooling choice is forced by the encoder type, not picked from a menu. Encoder = mean. Decoder = last. Optimized = attention.
The three standard pooling strategies
Mean pool (encoder-only default)
Given token outputs [h_1, ..., h_n] and attention mask m, mean pool computes the masked average. Works for encoders because bidirectional attention means every position has seen the full context; averaging is a coherent aggregation.
Used by: BGE-large-en-v1.5, BGE-M3, E5-large-v2, multilingual-e5-large, mxbai-embed-large-v1, Snowflake Arctic Embed L 2.0, Nomic Embed v1.5. This is the dominant 2026 default for open-weight encoder embedders.
Last-token pool (decoder-only default)
Take the hidden state at the final real (non-padding) position. Required for decoder-only embedders because causal masking gives different visibility to different positions.
Used by: e5-mistral-7b-instruct, NV-Embed-v2, Linq-Embed-Mistral, GritLM-7B. Often paired with an instruction prefix that biases what the last token summarizes.
Learned attention pool (optimized)
Introduce a learned query vector q and compute the weighted sum:
The softmax is over the token dimension. The learned q lets the model up-weight informative tokens. Used by Jina v3 (with task-specific LoRA heads) and some Cohere variants. Quality lift is 1 to 2 percent on MTEB-style benchmarks; cost is one extra trained component.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- BGE-M3 from BAAI uses mean pool and is the dominant open-weight multilingual embedder in 2026.
- e5-mistral-7b-instruct (Microsoft, 2024) introduced the decoder-only last token with instruction pattern at scale.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhen would you train a custom attention pooling head on top of an existing encoder?
When you have a specific downstream task (e.g. legal document retrieval) where some token positions are systematically more informative and you have labeled data to learn the weighting. Often a LoRA-style head on top of a frozen encoder is enough.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Picking CLS as a standard sentence-embedding pool. It is the BERT-classifier default, not the SBERT-lineage default. SBERT explicitly measured and rejected it for sentence similarity.
60 second bullets to scan on the way to the call.
Which pooling strategies are encoder-only defaults vs decoder-only defaults
Why causal masking forces last-token pool on decoder-based embedders
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.