Identify the key difference between Word2Vec and SBERT style embeddings

**The contrast** is static versus contextual. Static means each word has one vector, period. Word2Vec, GloVe, and FastText all live in this regime. Train the model once, get a lookup table, and at inference time you just look up the word. The vector for 'bank' is identical whether the sentence is about a river or about interest rates. Contextual means the vector depends on the surrounding words. SBERT and the modern sentence-embedding family (E5, BGE, GTE, Voyage v3) take a whole sentence as input and run it through a transformer. The transformer's attention layers let every word's representation depend on every other word's. The final sentence vector therefore differs when the same word appears in different contexts. The two blanks in the question are the canonical labels: 'static' (or one of its synonyms: context-free, non-contextual, fixed) and 'contextual' (or context-dependent, context-aware, dynamic). Why it matters: real language depends on context to mean what it means. 'I deposited my check at the bank' and 'the river flooded the bank' contain the same token but completely different meanings. Static embeddings collapse this; contextual embeddings preserve it. For sentence retrieval and RAG, that difference shows up as large gaps on every benchmark.

**Static** versus **contextual** is the cleanest one-word framing of the architectural split between Word2Vec and SBERT. A static embedding is a function from a single token to a single vector, learned once and frozen. The function ignores everything outside the token itself. Word2Vec, GloVe, and FastText are all static. The famous demonstration was the linear analogy `king - man + woman \approx queen`: the structure of the static vector space captures co-occurrence regularities but not contextual meaning. A contextual embedding is a function from a sentence to a vector (or to a set of per-token vectors, one per position). The function processes the whole input together so attention can mix information across positions. SBERT pioneered the recipe of running a pre-trained transformer over the sentence and pooling the contextual hidden states into one sentence vector. The modern descendants (E5-large, BGE-M3, GTE-Qwen2, Voyage v3, Cohere Embed v4) all share this architecture and dominate every retrieval benchmark. Three consequences flow from the architectural difference. Polysemy resolves. The contextual representation of 'bank' near 'river' differs from 'bank' near 'interest'. Static embeddings cannot do this because they have no input to condition on. Word order matters. The transformer's positional embeddings tie attention to position, so 'dog bites man' and 'man bites dog' produce different sentence vectors. Static embeddings averaged into a sentence vector are bag of words and order-agnostic. The inference cost shifts. Static is a lookup (essentially free); contextual is a transformer forward pass (milliseconds per sentence). For production RAG the quality gain is worth the cost, which is why contextual sentence embedders are the universal default in 2026.