Private AI Behind the Boundary: An Architecture for Sovereign LLM Operations
The organizations with the most to gain from modern AI are, almost by definition, the ones least able to use it as offered. A defense contractor holding CUI, a hospital system holding PHI, a bank holding transaction flows, an agency holding investigative records — each sits on exactly the high-value, high-context data that makes a large language model transformative, and each is barred, by regulation or by judgment, from sending that data to a commercial inference API. The result in most such organizations is not caution but paralysis: AI adoption stalls at the pilot stage, sensitive data stays merely stored rather than usable, and the teams closest to the mission watch capabilities they cannot touch mature on the open market.
The dilemma is architectural, not legal
The standard framing — "we can't use AI because of compliance" — mistakes the constraint. Compliance does not prohibit inference; it prohibits disclosure. A large language model that runs on hardware you control, reads only data you hold, and answers only to credentials you issue creates no disclosure at all. The question is not whether regulated organizations can use LLMs — it is whether the operating pattern around the model is engineered for the boundary or against it.
The four planes of a sovereign AI architecture
The model plane. Open-weight models have crossed the threshold where private hosting is a capability decision, not a compromise. Model selection, quantization strategy, and serving infrastructure are sized to the actual workload — summarization, retrieval-grounded question answering, and structured extraction rarely require frontier-scale weights — and the entire serving stack deploys on-premises, in a private cloud, or inside an air-gapped enclave, with model updates moving through the same controlled transfer procedures as any other artifact.
The data plane. Retrieval-augmented generation is where private AI earns its keep: answers grounded in your documents, records, and institutional knowledge rather than the open internet's average opinion. The retrieval index is built inside the boundary, embedding models run locally, and — critically — need-to-know access control is enforced at query time: the model can only be grounded in material the requesting identity is entitled to see. An LLM must never become an accidental declassification engine.
The governance plane. A private model does not equal a governed one. Enterprise deployment requires the controls a security officer can sign: least-privilege access for AI systems and agents, human-in-the-loop gates for consequential actions, complete audit of every prompt and response involving sensitive material, and scoped, revocable credentials for every automated consumer. Governance is enforced at the systems of record the AI touches — never by trusting the client.
The operations plane. Inference infrastructure is infrastructure: it needs metrics, alerting, capacity management, provenance-tracked deployment, and a tested recovery path, inside the same boundary as everything else. A sovereign AI capability that depends on an external observability vendor has quietly re-imported the dependency it set out to remove.
What changes when the boundary holds
With the four planes in place, the risk calculus inverts. Data custody is preserved by construction, so use-case approval stops being a legal negotiation and becomes an engineering review. Teams move from asking "may we?" to asking "which workload next?" — and the organization's most sensitive data, precisely because it never leaves, becomes its most productive.
Wilkes & Liberty operates this architecture in production. The productized form is the Helios private AI platform; the engagement form — agentic systems engineered on customer-controlled infrastructure — is our Agentic AI Development practice. Both start from the same premise: the boundary is not the obstacle to enterprise AI. It is the enabling condition.