Pick the best 2026 stack for a self-hosted on prem healthcare deployment

MCQ·Hard·4.0 · 0·~1 min·Asked atPerplexity Phonepe Workday

Attempt it

On-prem healthcare deployments operate under HIPAA. HIPAA cares about where patient data lives and who has access. Any API that requires patient prompts to leave the hospital network to reach a classifier, like OpenAI Moderation or Lakera Guard, fails the test, because the patient data has now left the boundary. That constraint rules out two of the four options. Option A uses OpenAI Moderation and a hosted GPT endpoint, both of which receive patient prompts in the cloud. Option C uses Lakera Guard, which is also a hosted API. Both violate residency on the inline path. Option D, relying on the base model's RLHF refusals, is below the bar for any production deployment, and especially for healthcare. RLHF lowers the rate of bad outputs on typical prompts but is not a runtime control you can audit or configure. A regulator asking 'show me your safety controls' will not accept 'the model is trained to refuse' as an answer. Option B is the only stack where every inline component runs inside the customer's network. **Llama Guard 4** and **Prompt Guard 2** are open-weights Meta models you deploy on the customer's GPU cluster. **NeMo Guardrails** orchestrates the rails with NVIDIA's Colang DSL. Offline red-team evaluation (Patronus or an internal harness) can run on synthetic or anonymised data, so the residency rule does not apply.

**The deciding constraint is data residency.** HIPAA-covered patient prompts cannot leave the on-prem boundary on the inline serving path. That single rule eliminates every option that calls a hosted safety API for live classification. Option A fails because OpenAI Moderation lives in OpenAI's cloud and GPT-4 hosted endpoints route every prompt outside the boundary. Option C fails because Lakera Guard is a hosted classifier service. Option D fails because base-model RLHF refusals are not a configurable, auditable runtime control, regulators evaluate per-request enforcement, and 'the model usually refuses' does not pass. Option B is the only stack where every inline component is self-hostable: - **Llama Guard 4** as the broad safety classifier with multimodal coverage and a configurable taxonomy that supports custom medical-policy categories. - **Prompt Guard 2** as the fast inline injection detector for direct and indirect attacks. - **NeMo Guardrails** with Colang flows to orchestrate input rails, output rails, retrieval rails (sanitising untrusted RAG content), and action rails (gating tool calls on irreversible operations). - **Patronus or an internal eval harness** running offline against synthetic and anonymised data for continuous red-team coverage. The stack also lines up with the FedRAMP and SR 11-7 governance posture most healthcare enterprises run alongside HIPAA. Engineering cost is real, the customer's team owns classifier drift and update cadence, but the residency constraint makes it non-optional.

TL;DR

Llama Guard 4 plus Prompt Guard 2 plus NeMo Guardrails orchestration, all self-hosted; hosted classifiers fail HIPAA residency, and RLHF refusals alone are not a guardrail.

Memory aid

Easy to grasp

Imagine a hospital that cannot let patient notes leave the building. Now imagine someone offers a smart sorting service that lives in a different building and just needs you to mail every note over for sorting. You cannot use it, no matter how good the sorting is, because the rule is the notes never leave. On-prem healthcare with HIPAA is exactly that hospital. Cloud safety APIs are exactly that off-site sorter. The only stack that fits the rule is one where every classifier runs on the hospital's own machines. Llama Guard, Prompt Guard, and NeMo all ship in a form you can run inside the building, which is why they win the question.

Key concepts

Safety & Guardrails

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Picking the right safety stack for an on-prem healthcare deployment is a question with a definite answer in 2026, and the reasoning chain that gets you there is exactly what staff-level interviewers want to hear. The decision is dominated by a single constraint: HIPAA residency on the inline serving path. Once that constraint is named, the rest of the choices fall out mechanically.

This walkthrough names the constraint, walks through why each non-winning option fails it, builds out the components of the winning stack, and addresses the trade-offs an honest senior answer should surface.

Mental model: in healthcare on-prem, residency is the floor. Every safety question stacks on top. If a component sends patient data to a third-party network, it is out, no matter how good the brand or how cheap the contract.

Why HIPAA residency rules out hosted classifiers

What HIPAA actually requires

The HIPAA Privacy Rule and Security Rule together govern protected health information (PHI). The simplified version: PHI must not be disclosed to unauthorized parties, and any third party that processes PHI is a Business Associate who must sign a BAA and meet specific security and audit standards.

A hosted safety API, OpenAI Moderation, Lakera Guard, hosted GPT endpoints, is by definition a third party. Even with a BAA in place, the operational fact is that prompts containing PHI leave the customer's network during classification.

Why on-prem amplifies the constraint

For cloud-hosted SaaS healthcare products, BAA coverage with cloud providers (AWS, Azure, Google Cloud) is the standard path: data lives in the cloud anyway, and the contract layer governs the relationship. The on-prem case is different. The customer has explicitly chosen to keep all infrastructure inside the hospital network. The whole point of on-prem is that data does not leave the building.

Routing prompts to an external classifier, even one with BAA coverage, contradicts the deployment model the customer paid for. Healthcare CIO offices in 2026 reject this routinely.

Specific failures of each non-winning option

Option A: OpenAI Moderation plus hosted GPT-4. The hosted GPT endpoint receives every patient prompt. OpenAI Moderation routes prompts to OpenAI's classification service. Two separate egress paths, both leaving the hospital network. Fails residency twice.
Option C: Lakera Guard as the only safety layer. Lakera Guard is hosted only, there is no self-hosted distribution in 2026. The inline classifier sits in Lakera's tenancy. Fails residency once, plus fails defense in depth (a single layer is below the bar for production healthcare regardless of residency).
Option D: RLHF refusals only. Two failures. First, no inline control, the model is what it is. Second, no audit log: a regulator asking 'show me your guardrail decision for patient X's prompt' cannot be answered because there is no decision to log. RLHF is a training-time signal, not a runtime control.

Building the winning stack component by component

Audit logging and the dual-LLM pattern

Trade-offs an honest senior answer surfaces

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Epic and Oracle Cerner integrations running Llama 4 Maverick on-prem deploy Llama Guard 4 plus Prompt Guard 2 self-hosted as the inline safety layer.
Hospital systems using NVIDIA DGX servers commonly run NeMo Guardrails as the orchestration shell on top of Llama Guard, with Colang flows reviewed by the compliance team.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you handle a new medical-policy category not covered by Llama Guard 4's default taxonomy?

Llama Guard supports custom taxonomy extensions via fine-tuning or prompt-based policy injection. Fine-tune a LoRA adapter on a small labelled dataset for the new category; deploy as an additional inference path; gate behind a separate NeMo rail so failures are isolated. Cost is days of engineering, not quarters of vendor roadmap.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Picking a hosted API stack because the brand is recognisable, without checking that patient prompts would have to leave the on-prem boundary to reach the classifier.

60 second bullets to scan on the way to the call.

Why HIPAA residency rules out hosted safety APIs on the inline path
Which Meta models are open-weights versus hosted-only

Primary sources. Browse if you want the original framing.