Metrics

Select every metric that belongs on a serious multi-agent eval dashboard

Match each LLM latency metric to what it actually measures

TTFT: name the metric it captures and what dominates it

In benchmarks, tokens per second measures what exactly?

Throughput, for an LLM serving system: give the working definition

Pick the best definition: in LLM evaluation, a metric is _____.

A teammate asks what 'running evals' means in the LLM context. How would an engineer explain it without jargon?

What metrics would you instrument to evaluate the health of a production MCP integration?

What does a rising approval rejection ratio signal in a production MCP integration?

Explain why string match accuracy is systematically misleading for LLM evaluation

Why string match accuracy fails as an LLM evaluation metric