The most common pooling strategy for SBERT-style sentence embedding models is pooling, which averages the token-level vectors across all positions.
A transformer encoder produces one vector per input token. To get a single vector for a whole sentence, you need a **pooling** step that combines the per-token vectors.
There are several pooling choices. Take the CLS token vector (BERT classifier style), take the last-token vector (used by decoder-only embedders), use a learned attention pool, or average all the token vectors.
**Mean pooling** is the average option. You take the token vectors, add them dimension-wise, and divide by the number of tokens (often with a mask so padding tokens are not counted). The output is one fixed-dimensional vector.
SBERT (the foundational sentence-embedding paper from Reimers and Gurevych, 2019) compared CLS pooling and mean pooling on sentence similarity benchmarks and found mean pooling consistently won. That choice set the default for the encoder-based sentence-embedding family that followed: BGE, E5, Snowflake Arctic, mxbai-embed.
Mean pooling has no learnable parameters, is cheap to compute, and works well when the underlying encoder has been fine-tuned with a contrastive objective.
**Mean pooling** is the dominant pooling strategy in the SBERT-family encoder embedding family. The mechanism is straightforward: given token-level vectors `H = [h_1, ..., h_n]` from the encoder's last hidden state, the sentence vector is
$$v = \frac{1}{|\text{mask}|} \sum_i \text{mask}_i \cdot h_i$$
where the mask zeros out padding positions so they do not bias the average.
The choice is empirical, not theoretical. SBERT's original paper measured CLS pooling, mean pooling, and max pooling on STS benchmarks and showed mean pooling produced materially better sentence embeddings than CLS. The intuition is that BERT's MLM pretraining never explicitly trained the CLS representation for sentence-level meaning. Using CLS for sentence similarity is asking for behavior the model was not optimized for.
The pattern stuck. BGE, E5, Snowflake Arctic Embed L 2.0, mxbai-embed-large, Nomic Embed v1.5, and most encoder-based 2026 embedding models use mean pool as their default. Exceptions exist. Decoder-only embedders (e5-mistral, NV-Embed) use last-token pooling for causal-mask reasons. Some models like Jina v3 use learned attention pooling for a small quality lift. But mean pool remains the safe and common default for encoder-based sentence embeddings.
Quick reference
TL;DR
Mean pooling, averaging the per-token vectors across all positions, is the SBERT-family default and remains the most common choice for encoder-based sentence embeddings.
Think of it like averaging scores from a panel of judges. An encoder gives you one vector per token in your sentence, but you need ONE vector for the whole sentence. The simplest way to combine them is to average: take each dimension, add it up across all tokens, divide by the count. That is mean pooling. It treats every token as equally informative about sentence meaning. It is dumb but it works, and SBERT made it the default for sentence embeddings.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
The encoder is decoder-only with causal masking (e.g. an LLM-based embedder) Token 0 has seen only itself under causal masking, mean pooling mixes vectors with wildly different context coverage and distorts the sentence representation.
The model was explicitly trained with CLS-pooling head (classification fine-tunes) If the model was trained to put summarization signal in CLS, using mean pool throws away that training. Match the pooling to what the model expects.
Token importance varies wildly within the input (long documents with boilerplate) Uniform averaging gives equal weight to a footer disclaimer and a key sentence; a learned attention pool can up-weight the informative tokens.
Quick scan · 2–4 min
Everything important, quickly.
6 to 7 min: mean pool formula with mask + empirical comparison vs CLS + SBERT-family adoption + decoder-only exception + learned attention pool variant.
Real-world examples
Real products, models, and research that use this idea.
1SBERT (Sentence-BERT) by Reimers and Gurevych (2019) established mean pool as the standard for sentence-similarity embeddings.
2BGE-large-en-v1.5 and BGE-M3 from BAAI use mean pooling as the default in 2026 production stacks.
3Snowflake Arctic Embed L 2.0 and mxbai-embed-large-v1 both default to mean pool on top of a BERT-family encoder.
4OpenAI text-embedding-3-small and 3-large abstract pooling behind the API but produce normalized vectors consistent with mean pool derived semantics.
5e5-mistral-7b-instruct and NV-Embed (NVIDIA, 2024) break the pattern with last-token pool because the underlying decoder uses causal masking.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat is attention pooling and when does it beat mean pool?
Sign in or go Pro to see the approach for this follow-up.
QWhy does forgetting the attention mask in mean pool produce a length-sensitive bug?
Sign in or go Pro to see the approach for this follow-up.
QCould you train a strong sentence embedding model on CLS instead of mean pool?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Confusing mean pooling with CLS-token pooling. SBERT explicitly compared both and showed mean wins for sentence similarity; CLS is the BERT as classifier default, not the sentence-embedding default.