Latency Cost Quality Tradeoff

When should automated eval escalate to human review in a production LLM product?

Which user-visible failure modes should trigger escalation from automated eval to human review?

How do you navigate the latency, cost, quality triangle when choosing an eval approach?

Order the three eval approaches from cheapest/fastest to most expensive/slowest