Why scan user input for PII before it reaches the model rather than only the output?
PII in the prompt is already at risk because trace exporters, prompt caches, provider logs, and eval pipelines persist the raw input long before any output rail fires.
Imagine writing your social security number on a postcard and dropping it in a mailbox. By the time the recipient reads it and decides whether to keep or shred it, the postcard has already passed through the postal sorter, the truck driver, the local office, and possibly a scanner that takes a photo of every piece of mail. Checking what comes back from the recipient does nothing for all the eyes that already saw the front of the card. A prompt sent to an LLM passes through similar hands, your tracing vendor, the provider's logs, a prompt cache, an evaluation pipeline, before any check on the model's reply has a chance to run. If you want the sensitive information to stay private, you have to redact it on the way out, not on the way back.
Detailed answer & concept explanation~9 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
2 min: prompts fan out to tracing, caching, provider logs, and eval pipelines before any output rail runs; input scrub keeps the compliance perimeter narrow; output scrub catches what input misses (model echoing or generating PII); compose both with provider data-retention configuration and tracing-side redaction.
Real products, models, and research that use this idea.
- Microsoft Presidio in 2026 ships PII detectors for 25+ entity types and integrates with NeMo Guardrails as an input rail.
- Anthropic's Claude API supports a 'zero data retention' tier for paying customers where the provider does not log prompts; enabling it is part of compliance configuration for HIPAA-covered workloads.
- Langfuse and Phoenix tracing offer attribute-level redaction so the tracing backend never stores raw PII even when the application forgets to redact at the edge.
- AWS Comprehend's PII detection runs as a sub-100ms inline service, commonly wired into application gateways as an input rail before the LLM call.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does prompt caching specifically change the PII threat model?
QWhere do you draw the line between redaction and blocking when input PII is detected?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Assuming the output guardrail is the only place PII can leak. By the time output runs, the input has already been copied to several systems outside your direct control.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.