When should an output guardrail fail closed vs fail open

**Fail-closed** means: if the output guardrail times out or errors, block the response and return a templated apology or a human-handoff message. **Fail-open** means: if the guardrail fails, release the model's response anyway. There is no universally right answer. The right answer is per-route policy: each user-facing surface in your product gets its own default based on the cost of a wrong release vs the cost of a wrong block. For a **medical-advice route**, the cost of a wrong release (an unmoderated hallucinated dosage instruction reaching a user) is catastrophic, regulatory, reputational, and human safety harm. The cost of a wrong block (the user sees an apology and tries again) is annoyance. Fail closed. For a **recipe-suggestion route**, the cost of a wrong release is mild, the model's own RLHF refusal already blocks the obvious problems. The cost of a wrong block is real UX damage on a high-traffic surface. Fail open. The EU AI Act sorts use cases into risk tiers; high-risk applications (medical devices, credit scoring, biometric categorisation) are required by law to have safety controls that fail closed. Whatever you choose, log every fail-open event. The log is the evidence for the incident review when a fail-open turns out to have released something bad.

**Fail-closed vs fail-open is a per-route product policy decision**, not a global toggle. The right framing: for each user-facing surface, ask 'what is the cost of a wrong release, and what is the cost of a wrong block?' **Default to fail-closed when:** - The route handles regulated content (medical, legal, financial advice). - The application is classified high-risk under the EU AI Act (credit scoring, biometric ID, recruitment screening, critical infrastructure, education access). - The blast radius of a single bad response is large (compliance violation, user safety harm, viral reputational risk). - The model's own RLHF refusal is not a credible backstop because the failure mode is subtle (hallucinated citations, off-policy advice phrased politely). **Default to fail-open when:** - The route is read-only, low-stakes, and high-traffic (FAQ answers, recipe suggestions, casual chat). - The model's RLHF refusal reliably catches obvious problems and the guardrail is a defence in depth backstop, not the primary defence. - Unavailability has a meaningful UX cost (every blocked response is a degraded user experience). **Operational rules.** - Encode the policy in a config file (`route_id, fail_mode, alert_threshold`). Per-route flags propagate at runtime. - Log every fail-open event with the original guardrail error, the released response, and trace ID. The log seeds the incident review and the regression eval. - Set alert thresholds: a sustained 10%+ fail-open rate on a route is an infrastructure incident even if no bad output has surfaced yet. - Build a templated fail-closed response that reads as graceful, not error-ridden. 'I need a moment to double-check that, please try again in a few seconds' is better than '500 Internal Server Error'. **The wrong answer is a global default.** Teams that pick one global mode are either over-blocking the cheap-traffic surfaces (fail-closed everywhere) or under-protecting the high-stakes ones (fail-open everywhere).