What is wrong with saying embedding dimension 47 is sentiment?

MCQ·Easy·4.0 · 0·~1 min·Asked atCohere Meesho·Relevant atHugging Face

Attempt it

People often hear that a sentence becomes a list of 1024 numbers. They then assume each number measures something: "this one is sentiment, this one is topic, this one is length." That intuition is wrong. The **dimensions are an arbitrary learned basis** (coordinate system). The training procedure had no instructions about what each coordinate should mean. It only adjusted the model parameters. The right pairs of inputs ended up close in vector space, and the wrong pairs ended up far apart. If you trained the same model again from a different random seed, you'd get a completely different arrangement. It would still work just as well for similarity search. Semantic concepts ARE encoded in the vectors. A sentiment classifier trained on top of frozen (fixed, not retrained) embeddings DOES find sentiment information. But the sentiment direction is a particular weighted combination of all 1024 dimensions, not any single coordinate. The same is true for topic, formality, and every other property. You cannot "make the sentiment positive" by editing one number in the vector. You cannot "filter by topic" by sorting on a single dimension. Treat the vector as one whole, and change it with operations on the full embedding.

**Embedding dimensions don't carry meaning on their own.** Semantic concepts ARE in the vector, but they live as directions (linear combinations across many or all dimensions) not as individual coordinates. Three pieces of evidence make this concrete. First, the **random-seed test**: train the same architecture twice with different initialisations and you get two embedding spaces where dimension 47 means completely different things. Both spaces support the same similarity search and downstream classifiers. That alone rules out any built-in coordinate semantics. Second, **probing classifiers**. When researchers train a linear probe to find sentiment in BERT embeddings, the result is a weight vector across all 768 dimensions. It is not a one-hot on any single one. The same holds for topic, syntactic role, and language identification. Third, **concept steering**. Concept editing in LLMs finds directions in activation space (compute the mean difference between contrasting pairs). Steering then applies a multiplier to that direction. Nobody steers by editing dimension 47, because dimension 47 doesn't mean anything in isolation. The other options fail. Option A imposes false structure. Option C implies human-readable dimensions for each model, which still aren't there. Option D treats training as encryption, which it isn't.

TL;DR

Embedding dimensions are an emergent learned basis with no inherent meaning; semantic concepts live in directions (linear combinations), not individual coordinates.

Memory aid

Easy to grasp

Imagine asking ten people to describe a face using exactly one hundred sliders. Each person picks their own meanings for their sliders. One uses brightness, another uses age, another uses kindness. Every face gets a unique slider setting, and faces that look alike end up with similar settings. But slider number forty seven does not mean the same thing to person A as it does to person B. They each made up their own system in private. The number-slots used to describe a piece of text work the same way. The system invents its own slider set during practice. The sliders are not sentiment, topic, or age. They are whatever pattern made the practice scores come out right. Meaning is real but it lives in combinations across many sliders at once, never on any single slider alone.

Key concepts

Embeddings

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

The intuition that dimensions must be features is the obvious extrapolation from how tabular data works. Acting on that intuition causes problems.

The random-seed argument

The cleanest test of "are dimensions meaningful?" is the random-seed experiment. Take the same model architecture, the same training data, the same loss function. Initialise the embedding matrix with two different random seeds. Train both to convergence.

What you get: two embedding models that produce different vectors for the same input but largely agree on similarity rankings. Cosine similarity between "cat" and "dog" will be high in both spaces. The absolute coordinates of "cat" will be completely different. A linear probe trained for sentiment on model A's outputs will find a different weight vector than the same probe on model B's outputs. Yet both probes will reach about the same accuracy.

If dimension 47 had a built-in meaning, this couldn't happen. Both models would have to converge on the same coordinate assignment for that meaning. They don't. The bases are unrelated because nothing in the training objective specifies them. The loss only constrains the geometry of distances and angles, not the absolute orientation of the space.

Where concepts actually live: directions

Superposition and polysemanticity

Why each wrong option fails

Practical implications

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Mechanistic interpretability work on Llama and Claude reveals superposition: concepts spread across dimensions, dimensions across concepts.
Sparse autoencoders on Anthropic models extract concept-aligned features but only after explicit decomposition. The raw dimensions are not concepts.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhat does mechanistic interpretability reveal about how concepts ARE encoded?

Concepts live as nearly-orthogonal directions in superposition: networks compress more features than dimensions by exploiting high-dimensional geometry. Sparse autoencoders can decompose activations into mostly-monosemantic feature directions, but the original dimensions themselves remain polysemantic.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Trying to "increase the sentiment" of an embedding by adjusting one coordinate. That coordinate doesn't represent sentiment. Semantic features are spread across the entire vector.

60 second bullets to scan on the way to the call.

Explain why dimensions are an arbitrary learned basis.
Define what "semantic concepts live as directions" means.

Primary sources. Browse if you want the original framing.