Drag each answer to line up with its matching prompt
Cohere Rerank 3.5
Late-interaction model, token-level scoring rather than sentence-level
Voyage rerank-3
Hosted multilingual cross-encoder behind a paid API, with no GPU to run
BGE Reranker v2 (BAAI)
Open weights, self-hostable, sizes from the 0.6B m3 model to 2B-class Gemma and MiniCPM variants
Jina Reranker v2
Hosted generalist reranker with a 32K-token input limit, from the vendor of the Voyage embedding models
ColBERT v2 / PLAID
Multilingual open weights under a non-commercial CC BY-NC 4.0 license
Cohere Rerank 3.5 is a hosted multilingual API; Voyage rerank-3 is hosted with 32K inputs; BGE v2 is open and self-hostable; Jina v2 is open but non-commercial; ColBERT uses token-level late interaction.
Imagine five careful interviewers, each with a specialty. The first one is the fast-talking generalist you hire from an agency. The second also comes from an agency and can read very long resumes in one go. The third is freelance and you can hire them onto your own team. The fourth is also freelance and reads resumes in many languages, but only works on non-commercial projects unless you sign a separate deal. The fifth has a completely different style: they read the resume word by word instead of skimming the whole thing. All five interviewers re-rank a small shortlist, but each has a personality and a quirk worth knowing.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
The reranker landscape has five offerings worth knowing. Four of them are sentence-level cross-encoders that differ in hosting, license, and input limits. The fifth is architecturally different and trades storage for token-level matching granularity.
Cross-encoder vs late-interaction architecture
Sentence-level cross-encoders (Cohere Rerank 3.5, Voyage rerank-3, BGE Reranker v2, Jina Reranker v2) read the query and chunk together as one input sequence. They produce one relevance score per pair via attention across the joint sequence. Each (query, chunk) pair takes one forward pass. The design is conceptually simple, fits standard transformer serving, and is well-supported across vector databases.
ColBERT v2 / PLAID is a late-interaction model. Each chunk is encoded into a sequence of per token embeddings, stored persistently in an index alongside the chunk text. At query time the query is also encoded per token. The relevance score is the MaxSim aggregation: for each query token, find its maximum similarity to any chunk token, then sum those maxima.
What this buys
It buys granularity. A sentence-level cross-encoder collapses the whole chunk into one score. So a perfect match on one sentence and weak matches everywhere else can produce the same score as moderate matches everywhere. Late-interaction sees the per token alignment directly and can credit the perfect-match sentence appropriately.
What it costs
It costs storage. Chunks need one embedding per token (often hundreds per chunk) rather than one chunk embedding. ColBERTv2 compresses each token vector, but the index is still much larger than one vector per chunk. PLAID is the optimized serving system that uses quantization and centroid pruning to make late-interaction serving practical at scale. The operational complexity is still higher than a sentence-level reranker.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Reranker | Hosting | Architecture | Distinguisher |
|---|---|---|---|
| Cohere Rerank 3.5 | Closed hosted API | Sentence-level cross-encoder | Multilingual hosted default, 4K-token input |
| Voyage rerank-3 | Closed hosted API | Sentence-level cross-encoder | Generalist, 32K-token input |
| BGE Reranker v2 | Open / self-host | Sentence-level cross-encoder | v2-m3 is Apache 2.0; sizes from 0.6B to 2B-class |
| Jina Reranker v2 | Open weights, non-commercial | Sentence-level cross-encoder | CC BY-NC 4.0, multilingual, 278M parameters |
| ColBERT v2 / PLAID | Open / self-host | Late-interaction (token-level) | MaxSim token scoring, finer granularity at storage cost |
Real products, models, and research that use this idea.
- Cohere Rerank 3.5 is available through Cohere's API and Amazon Bedrock, so a team can add a reranking stage without running a GPU.
- Voyage AI's rerank-3 and rerank-3-lite accept 32K-token inputs, so long chunks can be scored without splitting them first.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhen is ColBERT's late-interaction architecture worth the storage cost?
It is worth it for long chunks where sentence-level scoring loses information, or for queries with rare-token alignments. Run a head to head eval against a cross-encoder before committing to the storage overhead.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating all five rerankers as interchangeable. Architectural differences (cross-encoder vs late-interaction), hosting models (closed API vs open weights), licenses, and input limits each change which one is right for a given workload.
60 second bullets to scan on the way to the call.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.