Order the layers of a defence in depth safety stack from innermost to outermost

Order steps·Medium·4.0 · 0·~1 min·Asked atAnthropic OpenAI Microsoft

Attempt it

1RLHF / DPO / Constitutional AI preference training
2Pretraining data filtering and curation
3Runtime output guardrails (toxicity, redaction, structure)
4Audit logging and human escalation
5Runtime input guardrails (injection, PII, off-topic)
6Behavioural guardrails on tool calls (allowlist, confirmation)
7Supervised fine-tuning on safe demonstrations

**Defence in depth** for an LLM means stacking safety controls at different layers of the request path so that no single failure removes safety. The seven layers, from innermost to outermost: 1. **Pretraining data filtering and curation.** Decisions made about what data the base model sees. Removes the highest-risk content (CSAM, malicious code, private data) from the foundation. 2. **Supervised fine-tuning on safe demonstrations.** Teaches the model basic refusal patterns and helpful honest harmless behaviour using curated demonstrations. 3. **RLHF, DPO, or Constitutional AI preference training.** Aligns the model's preferences using human or model-based preference signals. Shapes the model's prior over outputs on typical inputs. 4. **Runtime input guardrails.** Live code at serving time that inspects every prompt for injection (Llama Prompt Guard, Lakera Guard), PII or secrets, off-topic content, and known jailbreak patterns. 5. **Runtime output guardrails.** Live code that inspects every model response for toxicity (Llama Guard 4, OpenAI Moderation), PII leakage, structural correctness, or citation requirements. 6. **Behavioural guardrails on tool calls.** Code around the orchestrator that enforces a tool allowlist, validates arguments, requires confirmation for irreversible actions, and bounds the number of steps an agent loop can take. 7. **Audit logging and human escalation.** Every rail decision and tool call is recorded immutably; ambiguous or high-risk cases route to a human reviewer. Layers 1-3 live in the weights. Layers 4-7 live in code around the model. The innermost layers reduce the base rate of unsafe behaviour; the outermost layers bound blast radius and produce the audit trail compliance regimes require.

The **defence in depth safety stack** for an LLM has seven layers running from training-time data curation outward to runtime audit logging. The ordering is innermost to outermost in the sense of which layers are baked into the model and which surround it at serving time. **Training-time layers (baked into weights):** 1. **Pretraining data filtering and curation.** Removes high-risk content (CSAM, deliberately malicious examples, leaked private data) from the foundation. This is the deepest layer; everything else operates on the resulting model. Modern frontier teams (Anthropic, OpenAI, Google, Meta) all run multi-stage filtering pipelines. 2. **Supervised fine-tuning on safe demonstrations.** Teaches refusal patterns and helpful behaviour through curated input-output examples. Sets the prior for the alignment phase. 3. **RLHF, DPO, or Constitutional AI preference training.** Aligns the model's preferences using human preference signals (RLHF), direct preference optimisation (DPO), or model-based critique against a constitution (Anthropic's Constitutional AI). Shapes the model's prior on typical inputs. **Runtime layers (code around the model):** 4. **Runtime input guardrails.** Llama Prompt Guard, Lakera Guard, Protect AI Rebuff, secrets-scanners, PII detectors, off-topic gating, jailbreak classifiers. Run before the model sees the prompt. 5. **Runtime output guardrails.** Llama Guard 4, OpenAI Moderation, AWS Bedrock Guardrails content filters, structured-output validators, citation checkers. Run after the model emits a response and before delivery. 6. **Behavioural guardrails on tool calls.** Tool allowlist enforced in orchestrator code, argument validation, confirmation gates on irreversible actions, max-step caps, per-tool rate limits. 7. **Audit logging and human escalation.** Every rail decision and tool call recorded immutably (S3 object lock, equivalent). Ambiguous or high-risk cases routed to human reviewers.

Quick reference

TL;DR

The defence in depth stack runs from training-time data curation through SFT, RLHF, runtime input rails, output rails, behavioural rails on tools, and finally audit logging, innermost (weights) to outermost (logs).

Memory aid

Easy to grasp

Imagine the safety layers of a castle. The deepest layer is the recruitment process for guards, who you let into the castle in the first place. Then comes their training. Then their drills and code of conduct. Out at the gate sits the door guard who checks anyone coming in. Past the door, a second guard checks anything leaving the castle. Around any tool the guards use (keys, weapons), a quartermaster signs items in and out. Finally, a scribe records everything in a ledger. Each layer is independent. If a clever visitor talks past the door guard, the exit guard might still catch them. If both fail, the ledger lets a magistrate review what happened.

Key concepts

Safety and Guardrails Pretraining

Detailed answer & concept explanation

~5 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

6 minutes: the seven layers in order, why they have independent failure modes, where the trust gradient runs, and how compliance regimes consume the outer layers as evidence.

Real-world examples

Real products, models, and research that use this idea.

Anthropic Claude Opus 4.7 production deployments combine Constitutional AI alignment with AWS Bedrock Guardrails input/output rails and Bedrock Agents action-group allowlists, all logged to CloudWatch with S3 object-lock retention
OpenAI GPT-5.5 deployments pair the model's RLHF refusals with OpenAI Moderation as input and output rails, plus tool-list allowlists in Assistants v2, plus customer-side audit logs
Meta Llama 4 Maverick reference deployments stack Llama Guard 4 (output toxicity) with Llama Prompt Guard (input injection) and a developer-side tool allowlist
AWS Bedrock Agents publishes a reference architecture combining Bedrock Guardrails (input/output), action groups (tool allowlist), and Lambda-side argument validation, with CloudTrail audit logging

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QIf you had to drop two layers from the seven for a constrained MVP, which two would you drop and why?

QHow does the layering change for a multi-agent system where one agent calls another?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Skipping the runtime layers because the training-time alignment looks strong, or skipping the audit layer because the rails seem to work, defence in depth requires independent failure modes.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.