**Input PII scanning matters because the raw prompt touches more systems than just the LLM.**
When a user submits a prompt containing 'my SSN is 123-45-6789, can you help with my tax question,' here is the realistic path of those bytes BEFORE the model generates a single token:
1. Your application logs the request for debugging.
2. Your tracing vendor (Langfuse, Phoenix, Datadog) records the prompt as a span attribute.
3. Your prompt cache (if you have one) stores the input keyed for reuse.
4. The provider (Anthropic, OpenAI) receives the prompt and may log it on their side per their data-retention policy.
5. Your async eval pipeline samples the trace later for quality scoring.
All five touchpoints happen before any output guardrail has a chance to act. The output rail can scrub the model's response perfectly and still the SSN is in your tracing vendor's database, in your prompt cache, and possibly in the provider's logs.
**Input PII scanning** redacts or blocks at step zero, before the prompt ever leaves your edge. The SSN gets replaced with `[REDACTED-SSN]` or the request gets rejected with an explanation. The downstream systems never see the raw value.
Output PII scanning is also valuable but solves a different problem: the model generating or echoing PII it should not. Both are needed; input scanning protects data the user supplied, output scanning protects data the model produces.
**The reason input PII scanning is not redundant is that the raw prompt fans out to several systems long before any output guardrail fires.**
Trace exporters, prompt caches, provider-side logging, and async eval pipelines all touch the prompt as data, not as an LLM input. Each of those is a system you may or may not control, with its own retention policy, its own access controls, and its own breach exposure. By the time the output rail runs, the PII has already been replicated to multiple stores.
**Concrete fan-out for a typical 2026 stack:**
- **Tracing.** Langfuse, Phoenix, Datadog, or an OTel collector records the prompt as a span attribute. Retention is commonly 30-90 days.
- **Prompt cache.** OpenAI's prompt caching and Anthropic's prompt caching both key on prompt prefixes; the prefix is held provider-side.
- **Provider logging.** Per the provider's data policy. Anthropic's default retention is short for paying customers; OpenAI's varies by plan. The 'zero data retention' tier exists but is opt-in.
- **Eval and analytics pipelines.** Sampled traces flow into evaluation infrastructure, often in cloud storage with broader access than the production trace store.
- **Application logs.** Your own structured logs, depending on log-level discipline, often capture the prompt.
All of these are upstream of the model output. None of them is in scope for an output guardrail.
**Input PII scanning** intervenes at the edge: redact, mask, or block. Microsoft Presidio, AWS Comprehend, Google DLP, and a long list of open-source PII detectors run at sub-100ms latency suitable for an inline rail. Redaction is usually preferable to blocking because it preserves the user's task, 'my SSN is [REDACTED-SSN], can you help with my tax question' is still a usable prompt, without leaking the value.
**Why this matters for compliance.** GDPR, HIPAA, PCI-DSS, and the EU AI Act all impose obligations on data once it touches a processing system.
Quick reference
TL;DR
PII in the prompt is already at risk because trace exporters, prompt caches, provider logs, and eval pipelines persist the raw input long before any output rail fires.
Imagine writing your social security number on a postcard and dropping it in a mailbox. By the time the recipient reads it and decides whether to keep or shred it, the postcard has already passed through the postal sorter, the truck driver, the local office, and possibly a scanner that takes a photo of every piece of mail. Checking what comes back from the recipient does nothing for all the eyes that already saw the front of the card. A prompt sent to an LLM passes through similar hands, your tracing vendor, the provider's logs, a prompt cache, an evaluation pipeline, before any check on the model's reply has a chance to run. If you want the sensitive information to stay private, you have to redact it on the way out, not on the way back.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Internal endpoints with no third-party data processors and a tracing stack you fully control If every system that touches the prompt is inside the same compliance perimeter, input scrubbing reduces fan-out exposure less; output rails may suffice.
Latency-critical paths where 30-80ms of inline PII detection breaks the SLO Move PII handling to a pre-screen step (separate API) the application calls before the chat endpoint; do not skip it entirely.
Endpoints where users are legally required to provide PII and redaction would break the workflow Tax-filing or healthcare-intake flows need the PII; the right answer is to scope retention and access, not redact.
Quick scan · 2–4 min
Everything important, quickly.
2 min: prompts fan out to tracing, caching, provider logs, and eval pipelines before any output rail runs; input scrub keeps the compliance perimeter narrow; output scrub catches what input misses (model echoing or generating PII); compose both with provider data-retention configuration and tracing-side redaction.
Real-world examples
Real products, models, and research that use this idea.
1Microsoft Presidio in 2026 ships PII detectors for 25+ entity types and integrates with NeMo Guardrails as an input rail.
2Anthropic's Claude API supports a 'zero data retention' tier for paying customers where the provider does not log prompts; enabling it is part of compliance configuration for HIPAA-covered workloads.
3Langfuse and Phoenix tracing offer attribute-level redaction so the tracing backend never stores raw PII even when the application forgets to redact at the edge.
4AWS Comprehend's PII detection runs as a sub-100ms inline service, commonly wired into application gateways as an input rail before the LLM call.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does prompt caching specifically change the PII threat model?
Sign in or go Pro to see the approach for this follow-up.
QWhere do you draw the line between redaction and blocking when input PII is detected?
Sign in or go Pro to see the approach for this follow-up.
QIf input scanning misses a novel PII format, how does the output scanner catch it?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Assuming the output guardrail is the only place PII can leak. By the time output runs, the input has already been copied to several systems outside your direct control.