Redact the offending span in place and forward the rest; dropping the whole response wastes work, passing through risks regulatory exposure, and retrying burns cost without fixing the root cause.
Imagine a letter that turns out to have one sensitive sentence in the middle. You could shred the entire letter, but then the recipient loses the helpful parts that were fine. You could mail it anyway, but you might get sued. The sensible move is to take a marker, black out the one sensitive sentence, and send the rest. That is span-level redaction. The recipient still gets value from most of the letter, the sensitive bit never reaches them, and you keep an unredacted copy in a locked drawer for the security team to investigate why it appeared in the first place.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
The output guardrail detected a phone number. What now? The reflex answer is to drop the whole response and call it safe. The production answer is more careful: span-level redaction preserves nearly all of the response's value while neutralising the specific leak. The wrong answers, full drop, pass-through, or retry, each fail on a different axis: UX, safety, or root-cause discipline.
This walkthrough explains why redaction is the right default, what marker style to use, how the audit pipeline preserves forensic capability without re-leaking the original value, and the edge cases (streaming, fully-leaked responses, regulated regimes) where redaction is not enough on its own.
Mental model: the output rail is a scalpel, not a hammer. Locate the offending characters precisely, replace them, log the original to a secured store, forward the rest. Full-drop is reserved for irrecoverable cases.
Why span-level redaction wins the trade-off
The UX cost of full-drop
The response the user sees when the rail drops the whole answer is some generic error: 'I cannot provide that response right now.' For a model that just produced 400 words of useful information with 11 characters of accidental PII in the middle, the user loses everything. Repeated full-drops on a route train users to associate the product with refusals, which is exactly the wrong learned behaviour.
In A/B tests at any meaningfully-scaled product, full-drop on PII hits drives measurable drops in engagement and increases in churn. The marginal safety gain over redaction is approximately zero: in both cases, the PII does not reach the user.
The safety cost of pass-through
The opposite extreme, forward the response unchanged and flag for offline review, fails on safety. Offline review is a forensics tool, not a guardrail. By the time the human reviewer reads the log, the user has already received the PII. For regulated content, the disclosure has already triggered audit obligations. Pass-through with flag is the right pattern for low-confidence borderline cases, not for confirmed PII detection.
The cost discipline issue with retry
Retrying the model call on detection burns a second inference (cost), adds latency (200 ms to 2 seconds), and does not address why the PII appeared. If the cause is in the prompt context or retrieval, the second call produces the same leak with high probability. If the cause is random sampling and the second call happens to be clean, the team learns nothing about how to prevent the first call's leak. Retry as a guardrail is not a strategy; it is wishful thinking.
Why redaction is the scalpel
The detector identifies exactly which characters constitute the PII. The redactor replaces those characters with a marker and forwards the rest of the response unchanged. The user gets a coherent answer with one obviously-replaced span; the regulatory exposure is neutralised; the original value is preserved in audit. The UX cost is bounded to the visible marker; the safety guarantee is the same as full-drop.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- AWS Bedrock Guardrails ships native PII redaction modes with both opaque ([REDACTED]) and type-tagged ([PHONE]) output formats, configurable per policy.
- Microsoft Presidio is the open-source canonical PII redaction library, widely used as a rail in NeMo Guardrails deployments on top of Llama 4 and Claude Opus 4.7.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you architect PII redaction in a streaming response so detection fires before characters reach the client?
Buffer the stream in small windows (50-200 tokens) and run the detector at each flush; only release the window once detection passes. Adds latency proportional to the buffer size. Alternative: speculative streaming where the client receives tokens optimistically and a separate signal commits or revokes the window. Most production stacks pick fixed-window buffering for the simpler reasoning.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Dropping the entire response on every PII hit; the model may have produced a useful answer with one accidental span, and full-drop is a UX hammer where redaction is the scalpel.
60 second bullets to scan on the way to the call.
Span-level redaction versus full-response drop trade-off
Three redaction marker styles (opaque, type-tagged, structural placeholder)
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.