Define an embedding in terms of what it IS, not what it does

Attempt it

Your answer

Quick reference

TL;DR

An embedding is a learned dense vector of fixed length whose position in space encodes semantic content: proximity equals similarity.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine every sentence gets pinned onto a giant map. Sentences that mean similar things get pinned close together; unrelated ones land in different neighbourhoods. "The puppy is barking" and "my dog is loud" sit next to each other; "the cake is tasty" sits across town. Once everything has a pin, comparing meanings is just measuring how far apart the pins are. The clever part isn't the map; it's the program that chose the pin locations. It read enormous amounts of text and learned to place new pins so neighbours actually share meaning. After that, sorting, searching, and grouping text becomes simple distance math instead of guessing what words mean.

Key concepts

Embeddings

Detailed answer & concept explanation

~5 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

Define the data object (dense fixed-dim vector). Add the learned property (geometry encodes meaning). Explain why a random vector of the same shape is not an embedding. Close with the metric (cosine) and two production examples.

Real-world examples

Real products, models, and research that use this idea.

OpenAI's text-embedding-3-large emits 3072-dimensional dense vectors used in default RAG stacks across 2026.
Voyage v3 and Cohere embed-v4 are production text embeddings shipped via API, both built on transformer bi-encoders.
Open-source BGE-M3 and Jina-embeddings-v3 produce multilingual dense vectors with optional Matryoshka truncation for storage trade-offs.
Pinecone, Weaviate, Qdrant, and pgvector all assume the same definition: fixed-dim dense vectors with a similarity metric.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhat property does the training objective enforce that makes the geometry meaningful?

Sign in or go Pro to see the approach for this follow-up.

QWhy can't you mix embeddings from two different models in the same index?

Sign in or go Pro to see the approach for this follow-up.

QHow does Matryoshka representation learning change the definition?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Defining an embedding by listing what it's used for (search, RAG, clustering) rather than what it IS: a fixed-length dense vector with learned geometric structure.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.