Embeddings map discrete text into a continuous vector space where geometric distance corresponds to semantic similarity, turning meaning into math.
Imagine giving every sentence a spot on a giant map, where sentences that mean similar things end up near each other. "My dog is barking" and "the puppy is loud" would sit close by. "The cake is tasty" would sit far away on a different part of the map. Once every sentence has a spot, comparing meanings becomes as easy as measuring how far apart two pins are. That is what this idea does for text. It turns words into numbers a computer can line up on a map. You can sort, search, group, and pair up sentences by simple map distance. The trick is the program that picks the spots. It has read tons of text and learned how to place new sentences in a place that respects meaning. That way the math done on top matches what humans think feels close.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
The abstract definition of embeddings (dense vectors in a learned space) has concrete consequences (retrieval becomes geometry, RAG becomes possible, similarity becomes measurable).
What embeddings are, concretely
An embedding is a vector. For a typical text embedding model, the vector has between 256 and 3072 floating-point components, depending on the model. Each component is just a number. The components together describe a position in a high-dimensional space.
That position is not random. An embedding model computes it. The model has been trained on huge amounts of text, usually with a contrastive objective. That objective pushes semantically related text pairs toward similar positions and pushes unrelated pairs apart. The model has learned a map: input text in, position in vector space out.
The map has a key property. Two pieces of text with similar meaning end up at nearby positions. Two unrelated pieces end up far apart. This property is what makes the embedding useful. It also means embeddings are lossy: the vector is built for similarity comparisons, not for reconstruction. Even so, inversion attacks such as Vec2Text can recover much of a short input, so an embedding does not hide the text.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenAI's text-embedding-3-large produces 3072-dimensional vectors and is the default embedding for many RAG stacks.
- Voyage AI offers separate domain models for code, finance, and law (voyage-code-3, voyage-finance-2, voyage-law-2) where general embeddings can underperform.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat is the difference between word embeddings (Word2Vec, GloVe) and modern sentence embeddings?
Word embeddings produce one fixed vector per word regardless of context. Sentence embeddings produce one vector per sentence or document, computed from a contextual encoder that handles polysemy and longer-range meaning. Word embeddings are static; sentence embeddings are contextual.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Confusing embeddings with compression, encryption, or translation. Compression is lossless; encryption is reversible by key; translation produces text. Embeddings produce dense vectors organised by meaning, not by any of those properties.
60 second bullets to scan on the way to the call.
Define embeddings as dense vectors that encode semantic content geometrically.
Identify nearest-neighbour search as the operation embeddings enable.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.