Order the three eval approaches from cheapest/fastest to most expensive/slowest

The right order is a cost ladder: cheapest and fastest first, most expensive and slowest last. So you start with **automated metrics**, then LLM-as-judge on a sample, then event triggered human spot-checks, and finally human expert evaluation. Automated metrics like format checks, regex, and string matching run in milliseconds on every output for essentially zero cost. They catch the obvious failures, broken JSON, missing fields, wrong language, before anything pricier looks at the output. LLM-as-judge sits next. A judge model scores quality on a sampled subset, not the whole set, because each call costs money and takes time. It catches things automation cannot, like helpfulness and tone. A periodic human spot-check comes after that. It only fires when the LLM-judge scores drift, so it runs occasionally rather than continuously, which keeps its cost moderate. Full human expert evaluation is last. Experts are accurate but slow and expensive, so you reserve them for the small slice that truly needs domain judgment.

The ordering is a cost and latency ladder, cheapest first. Automated metrics come first. Format checks, regex, and string matching run in milliseconds at near-zero cost. So you run them on every single output to filter obvious failures. LLM-as-judge on a sample comes second. Each judge call costs money and adds seconds of latency. You run it on a sampled subset, not the full set, to score quality dimensions automation cannot reach. The periodic human spot-check comes third. It is event-triggered: it fires only when judge scores drift, so it runs occasionally rather than continuously. That keeps human cost bounded while still catching judge miscalibration. Human expert evaluation comes last. Domain experts are the most accurate signal but the slowest and most expensive. You reserve them for the small slice needing real domain judgment, or as the periodic ground truth that recalibrates everything upstream. The principle is simple. Each tier is cheaper and broader than the next. Cheap tiers run on everything; expensive tiers run on a shrinking sample. Get the order wrong and you burn budget paying experts to catch broken JSON.