Flashcard: what is an embedding in a RAG pipeline, and what role does it play?

Flashcard·Easy·4.0 · 0·~30s

Attempt it

An embedding is a fixed-length numeric vector that represents text in a high dimensional space, produced by a model like text-embedding-3-large, Voyage Embed v3, or Cohere Embed v4. In RAG, you embed every document chunk once at indexing time and embed the user query at search time; the closer two vectors are by cosine similarity, the more semantically similar the texts. This is what lets retrieval find chunks by meaning instead of by exact keyword match. Embeddings are NOT learned per query and NOT the same as model weights: they are an artifact you compute and store, then look up at retrieval time.

Your answer

Quick reference

TL;DR

An embedding is a fixed-length vector that places text at a point in high dimensional space, so semantic similarity becomes vector distance.

Memory aid

Easy to grasp

Imagine every sentence in the world gets a numeric address, like coordinates on a giant map. Sentences with similar meaning live close together, even when they use different words. 'Cancel my subscription' and 'how do I quit my plan' end up as next door neighbors, while 'best pasta recipes' lives in a completely different neighborhood. A special address maker computer reads each sentence and writes down its address. In RAG, you give every chunk of your documents an address ahead of time and store all the addresses. When a user asks something, you compute the address of their question, then look up which document chunks live nearest. You are no longer searching for matching words. You are searching for matching meaning, which is the whole reason RAG can answer paraphrased questions at all.

Key concepts

Embeddings Retrieval-Augmented Generation Vector databases

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

3 min: define embedding as a vector address for meaning, walk through index-time and query-time encoding, name 2026-current models, and call out the same model and re-index rules.

Real-world examples

Real products, models, and research that use this idea.

OpenAI text-embedding-3-large producing 3072-dimensional vectors for RAG corpora over private documents and knowledge bases.
Voyage Embed v3 asymmetric models offering separate query and document encoders tuned for retrieval, used inside Anthropic's RAG stack.
Cohere Embed v4 multilingual embeddings powering retrieval across mixed language enterprise knowledge bases in production.
pgvector storing OpenAI or Voyage embedding outputs in Postgres for teams that want retrieval without a dedicated vector database.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy must query embeddings and document embeddings come from the same model?

QHow would you decide between 1024-dim and 3072-dim embeddings for a million chunk corpus?

QWhat is an asymmetric embedding model and why might it help retrieval?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating an embedding as part of the LLM's weights. It is a separate artifact computed by a separate model and stored ahead of time, not something the chat model produces during generation.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.