Match each retriever-evaluation metric to what it actually measures

These five metrics split into two groups, and the first job is to separate them. Four are retriever metrics. They judge whether the system fetched the right documents and how it ordered them, ignoring the final written answer. - **Hit rate** is binary per query: did at least one relevant document land in the top-k? You average the yes/no across queries. - **Recall@k** is a fraction: of all the documents that are relevant to a query, how many appeared in the top-k? - **MRR** cares about position. It takes the rank of the first relevant result, inverts it (rank 1 scores 1, rank 4 scores 0.25), and averages over queries. - **NDCG** is the most refined. It uses graded relevance (some docs are more relevant than others), discounts results that appear lower down, and normalizes against the best possible ordering. One is a generation metric. Faithfulness, scored by an LLM judge, ignores ranking entirely and asks whether the answer's claims are actually supported by the retrieved context. A perfectly retrieved set can still produce an unfaithful answer, so this measures a different stage.

The pairs split into retriever metrics and a generation metric, and getting that boundary right is the whole exercise. Four metrics judge **retrieval**, computed against labelled query to relevant document pairs. Hit rate is binary presence: did any relevant document make the top-k? Recall@k is graded presence: what fraction of all relevant documents made the top-k? MRR judges position by averaging the reciprocal rank of the first relevant result, so surfacing it at rank 1 beats rank 5. NDCG is the most complete: it uses graded relevance, applies a logarithmic position discount, and normalizes against the ideal ranking. So a highly relevant document placed first scores better than a marginally relevant one. The fifth, **faithfulness**, is a generation metric scored by an LLM judge. It ignores ranking and asks whether the answer's claims are entailed by the retrieved context. In other words, whether the model grounded its answer or hallucinated beyond what it was given. The reason the split matters is diagnostic. The four retrieval metrics tell you whether the right context reached the model; faithfulness tells you whether the model used it honestly. A system can score high on recall and MRR yet low on faithfulness, which points you at the generator, not the retriever. Mismatching faithfulness into the ranking group hides exactly that distinction.