What is an air-gapped LLM deployment and how does it work?

An air-gapped LLM deployment is an installation with no physical or logical connection to the public internet, used when a data classification or contract requires that a model and its data never touch an external network. It works by shipping the model weights, the inference engine such as vLLM or TensorRT-LLM, and any supporting containers into the isolated environment on encrypted physical media or through a one-way transfer diode, then installing and validating them entirely offline. GPU servers, storage and networking sit inside a segmented enclave, often with its own certificate authority, package mirror and monitoring stack, so nothing depends on reaching the outside world during normal operation. This approach is common in defense, government, critical infrastructure and some financial and healthcare environments where regulatory or classification rules mandate physical isolation, not just encryption. The trade-off is slower model and security updates, since every change requires a manual, audited transfer process rather than an automatic pull. Air-gapped setups also need local logging, local authentication and local backup, since no cloud service can be relied on for any of them. Nanobase AI, an NVIDIA Inception Program member, has designed and installed air-gapped GPU systems for exactly these high-assurance environments.

Air-gapping is a property of the whole stack, not just the model

An LLM itself does not need the internet to answer a prompt, but an air-gapped deployment has to remove every incidental network dependency the surrounding stack normally assumes exists, from container registries to license servers to monitoring agents that phone home. The hard part of air-gapping is rarely the model; it is auditing every component in the stack for a hidden network call and replacing or disabling each one before the system goes fully isolated. A container orchestrator that tries to pull an image from a public registry, or a monitoring agent that pushes metrics to a cloud dashboard, will fail silently or noisily inside an air gap unless it is reconfigured first.

The layered architecture

Every layer of the stack needs its own air-gapped equivalent of its normal network-connected behavior, from the container registry down to the identity provider.

Building an air-gapped deployment step by step

1. Stand up an internal container registry and package every image the stack needs, including the inference engine, chat UI, and any supporting services, before transfer.
2. Download model weights and all dependencies on a connected staging system, then verify checksums against the publisher's published hashes.
3. Transfer the full bundle, container images, model weights, and configuration, across the air gap using encrypted physical media or a certified one-way data transfer device.
4. Install and configure the identity provider, inference engine, and chat interface entirely within the isolated network, with no step assuming a live internet check.
5. Validate that no component attempts an outbound connection, using network traffic monitoring inside the isolated environment during a full test cycle.
6. Document the transfer and validation process, since most air-gap requirements come with an audit obligation, not just a technical one.

Following this sequence in order, registry first and validation last, prevents the most common failure mode: a component installed before its network dependency was identified and removed.

Physical and procedural controls matter as much as network isolation

A network with no internet route is necessary but not sufficient for a true air gap, since physical access controls, media handling procedures, and personnel vetting close the gaps that pure network isolation leaves open. Many air-gap failures in practice come from a laptop with dual network interfaces bridging the isolated network to the internet, or a USB drive carrying data in both directions without going through the controlled transfer process. Formal air-gap environments typically pair the technical isolation with strict media control policies and logged physical access to the hardware room.

Frequently asked questions

Does air-gapping mean no updates at all?

No, air-gapped systems can still be updated, just through a controlled physical transfer process rather than a network pull, covered in detail under updating models in an air-gapped environment. The system stays static only between scheduled transfer windows.

Is air-gapping only relevant for government and defense?

No, while government and defense are the most visible users, air-gapped or near-air-gapped deployments are increasingly used by financial institutions, critical infrastructure operators, and any organization whose contracts or internal policy prohibit any external network path for certain data classes.

How is air-gapped different from just blocking outbound traffic with a firewall?

A firewall rule is software that can be misconfigured or bypassed by a determined actor with internal access; a true air gap has no physical network path to bypass, which is a materially stronger guarantee for the highest-sensitivity environments, though it comes with real operational overhead.

How Nanobase AI helps

Nanobase AI designs and installs air-gapped LLM deployments end to end, including the internal registry setup, the verified transfer process for model weights and containers, and the fully offline monitoring and identity stack that keeps the environment genuinely isolated. The team also builds the recurring update pipeline so an air-gapped system does not fall permanently behind as models improve. See our on-premise deployment guide for the broader architecture this fits into.

Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.