Pick the architectural reason embedding models are bi-encoders rather than cross-encoders

An **embedding model** must do one thing at scale: let you search across millions of documents in milliseconds. That requirement is what forces the bi-encoder design. A bi-encoder runs the document through the encoder once, at index time, and stores the resulting vector in a database. Later, at query time, you encode just the query and look up nearest neighbors in the index. The expensive transformer work on the documents has already been done. A cross-encoder does not produce separate vectors. It takes the query and a candidate document, concatenates them, and runs a single transformer over the joint input. There is nothing to precompute, because the score depends on both sides together. Every new query means redoing the transformer pass over every candidate. For a million-document corpus, that is a million forward passes per query. No production system can pay that cost for first-stage retrieval. Bi-encoders are the only design that scales, so embedding models are bi-encoders. Cross-encoders show up later in the pipeline as rerankers over a small candidate set.

The architectural reason is **indexability**. A bi-encoder factorizes the scoring function as `s(q, d) = f(q) . g(d)`, so `g(d)` can be precomputed offline for every document and stored in a vector index. Query time becomes one encoder pass plus an ANN lookup, both fast. A cross-encoder has no such factorization. The score `s(q, d) = CrossEncoder([q ; d])` is a non-separable function of both inputs. The query is unknown at index time, so nothing about the document side can be precomputed in a form that finishes cheaply at query time. The other options are wrong for these reasons. Cross-encoders can be trained with contrastive losses, no problem. Cross-encoders are typically higher per-pair quality than bi-encoders, not lower, that quality gap is exactly why they are used as rerankers. And cross-encoders handle long inputs fine; modern rerankers like Cohere rerank-3.5 handle 4k to 8k token windows. The right mental model: bi-encoders trade per-pair quality for scalable indexing, cross-encoders trade scalability for higher per-pair quality. You use both in a two-stage retrieve and rerank pipeline.