FedRAMP limits you to the government SKUs (AWS GovCloud Bedrock, Azure Government OpenAI, GCP Assured Workloads Vertex) or self-hosted open weights, and every adjacent vendor (observability, gateway, vector DB) also
Imagine you can only buy ingredients from a tiny set of approved stores. The food might be similar to what's at the regular supermarket, but you cannot mix in things from anywhere else, and new items take a long time to be approved. FedRAMP is like that for federal cloud software. The government has a list of authorized cloud regions and vendors, and you can only build your LLM app from items on that list. The commercial OpenAI API, fast-moving startup observability tools, and most niche vector databases are not on the list. You end up using AWS GovCloud or Azure Government, the FedRAMP version of Bedrock or Azure OpenAI, and a smaller set of authorized supporting tools. The build is slower and the menu is shorter, but the certification is the price of doing federal business.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
FedRAMP authorization is the constraint that decides what your federal LLM stack can look like before any benchmark or model-choice conversation happens. A team accustomed to building on the commercial OpenAI API and a fast-moving LLMOps SaaS toolchain discovers, on the day a federal customer asks for an ATO package, that most of those tools are simply not available. The choice is not 'pick the best model' but 'pick from a small list of authorized endpoints and rebuild the rest of the stack from authorized parts'.
This walkthrough breaks down what FedRAMP is, the cloud government SKUs that carry LLM endpoints in 2026, the adjacent stack components that also have to be authorized, the operational trade-offs (model lag, pricing premium, multi-region constraints), and when self-hosted open weights become the only viable answer.
What FedRAMP is and why it dictates the inventory
FedRAMP (Federal Risk and Authorization Management Program) is the federal government's standardized cloud-service authorization program. A cloud service offering goes through a third-party assessment against the FedRAMP control baseline (Low, Moderate, or High impact), receives a Provisional Authorization to Operate (P-ATO) from the FedRAMP PMO or a direct ATO from a sponsoring agency, and then enters continuous monitoring with monthly vulnerability scans and annual assessments.
The practical effect is that a federal agency can only use a cloud service if that service has a FedRAMP authorization at the impact level matching the data being handled. Low covers public-facing unclassified workloads. Moderate covers most federal-civilian workloads with controlled unclassified information (CUI). High covers high-impact systems where loss of confidentiality, integrity, or availability would have severe or catastrophic effect.
Authorization is continuous, not one-time. A vendor that lets its monthly scans slip or fails an annual assessment can lose authorization. That continuous obligation is the reason many fast-moving commercial vendors never pursue FedRAMP: the operational discipline is incompatible with weekly feature shipping.
The boundary matters. FedRAMP authorizes a specific service inside a specific environment. AWS GovCloud (US) is a separate cloud region from AWS commercial; it has its own authorization. Azure Government is separate from Azure Commercial. Crossing the boundary (sending data from a GovCloud workload to a commercial endpoint) breaks compliance regardless of contract.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- AWS GovCloud (US) Bedrock ships Anthropic Claude and Amazon Nova at FedRAMP High for federal civilian and DoD workloads.
- Azure Government Azure OpenAI is the path most US federal agencies use for OpenAI access under FedRAMP High.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does FedRAMP High differ from Moderate for LLM workloads, and when do you need each?
High is required for high-impact data (national security adjacent, sensitive personally identifiable information at scale). Moderate covers most federal-civilian workloads. The control count is higher at High (around 421 vs 325) and continuous monitoring is more stringent. Most LLM workloads target Moderate; intelligence-adjacent or law-enforcement workloads target High.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Assuming a commercial vendor's HIPAA or SOC 2 posture maps to FedRAMP. FedRAMP is its own continuous authorization program with its own controls and its own boundary; only authorized vendors and SKUs count.
60 second bullets to scan on the way to the call.
The three cloud government SKUs that carry FedRAMP-authorized LLM endpoints
Why direct commercial API access is out of scope for FedRAMP workloads
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.