Describe Anthropic's built in safety filtering layer

Flashcard·Easy·4.0 · 0·~30s·Asked atMeesho Robust Intelligence Uber

Attempt it

Anthropic ships Constitutional-AI-tuned models plus a server-side safety filter that runs on every API request, it intercepts and refuses generations that violate the usage policy taxonomy. It is invisible to the developer except as an occasional refusal or moderation flag in the response. It does NOT customise to your deployment's specific policy, does not detect prompt injection per se, and does not redact PII you put into prompts. You still need a runtime guardrail layer for those.

Your answer

Anthropic's safety has two parts. The first is at training time. Claude models are tuned with Constitutional AI and RLHF so that the model itself refuses harmful requests most of the time without any extra filter in front of it. The second part is at serving time. Anthropic runs a server-side safety filter on every API request. The filter checks for content that violates Anthropic's published usage policy: things like CSAM, instructions for mass-casualty weapons, and other clearly disallowed categories. When the filter trips, the API returns a refusal or a moderation flag, and you see it as a regular response with content that explains the refusal. What the layer does not do is just as important. It does not know what your application considers off-policy, if you sell legal services, you decide what counts as legal advice, not Anthropic. It does not detect prompt injection, if a user pastes an attack, the filter is looking at the content categories, not at injection patterns. And it does not redact PII or secrets that flow through prompts. For those, you need your own input and output rails.

Anthropic's first-party safety is **two layers, neither of which is your custom policy**. **Layer one, model behaviour.** Claude is trained with Constitutional AI and Reinforcement Learning from AI Feedback, which produces a model whose default behaviour is already aligned to Anthropic's usage policy. For most categories of harm, the model refuses without any external filter being involved at all. **Layer two, server-side filter.** A classifier runs at the API edge on both the prompt and the generated completion. It checks against Anthropic's usage policy categories (e.g. CSAM, weapons of mass destruction, mass violence). When it trips, the response either contains a refusal or sets the appropriate moderation indicator. The filter is invisible to integrators except in its output; you cannot tune its thresholds, change its taxonomy, or whitelist categories. The boundary it does **not** cover is the entire surface of a real production app: your application-specific policy (what your product considers off-topic, off-brand, regulated), prompt-injection detection (the filter does not classify 'ignore previous instructions' as a harm category), and PII redaction (the filter does not strip the social security number a user pasted into a prompt). Those belong to a runtime guardrail layer you operate yourself, Llama Guard 4, Anthropic's own Prompt Guard, Bedrock Guardrails, NeMo Guardrails, Lakera Guard, or a combination. The first-party layer is a baseline, not the safety stack.

TL;DR

Anthropic ships training-time safety (Constitutional AI) plus a server-side filter on its usage policy taxonomy; it does not cover your custom policy, prompt injection, or PII redaction.

Memory aid

Easy to grasp

Think of a movie theatre that refuses to show certain movies as a house rule. That is what Anthropic does for Claude. They have trained the actor to behave well, and they keep a quiet bouncer at the door who refuses to perform scenes from a banned list. That bouncer is the same for every theatre in the chain. If your particular theatre also wants to ban a custom list, say no spoilers for the new release, the chain's bouncer will not do that for you. You need your own bouncer inside your own theatre. And the chain's bouncer is not watching for people who sneak in fake scripts that tell the actor to break character; that is a different job, and you have to hire someone for it.

Key concepts

Safety and guardrails RLHF and alignment

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

When teams pick a frontier model provider, one of the first questions they ask is what safety they get for free. The answer for Anthropic is more substantive than for many providers, but it is also more bounded than it appears at first glance. Treating Anthropic's first-party safety as the safety stack rather than as one layer is a recurring architectural mistake that surfaces in incidents months later.

This card splits the question into three pieces. First, what Anthropic actually ships at training time. Second, what runs at serving time. Third, the explicit gaps your runtime guardrails have to cover. By the end you should be able to draw a clean line between provider responsibility and integrator responsibility.

Training-time safety, Constitutional AI and RLHF

Claude is not a base language model with a content filter bolted on. It is trained with Constitutional AI, an Anthropic-developed method where the model is shown a written constitution of principles and learns to critique and revise its own outputs against those principles. The result is a model whose default behaviour already reflects Anthropic's usage policy without any external check.

The practical effect is that for many categories of harmful request, the model declines or steers the conversation on its own. You do not see a filter trip; you see the model behave appropriately.

Why this matters operationally

A trained-in disposition is more robust than an external filter for adversarial cases that depend on context. A filter has to classify text in isolation, but the model has the full context, system prompt, conversation history, user message, and can make contextual judgements. Anthropic chose this approach precisely because pre/post filters were brittle on long context.

Serving-time safety, the server-side filter

Three things the first-party stack does not cover

How to architect on top of the first-party layer

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Anthropic publishes Prompt Guard as a separate small classifier precisely because injection detection is not in the Claude API's built-in filter.
AWS Bedrock Guardrails exists in part to let teams layer their own taxonomy on top of Claude served through Bedrock, including custom denied topics and PII filters.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you architect a second-layer guardrail that complements Anthropic's first-party filter without duplicating its work?

Focus your layer on application-specific risks (custom topics, injection, PII, secrets, tool gating); let Anthropic's filter handle its usage-policy categories and treat it as a backstop.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Assuming Anthropic's edge filter covers prompt injection, custom policies, and PII; it only covers Anthropic's published usage policy categories, and your application-specific risks still need their own guardrail layer.

60 second bullets to scan on the way to the call.

Two layers in Anthropic's safety, training time and serving time
What Constitutional AI does to default model behaviour

Primary sources. Browse if you want the original framing.