You install the NVIDIA GPU Operator with Helm, adding the NVIDIA repository and running a single install command against a cluster that already has a working container runtime and, ideally, no GPU driver pre-installed on the nodes. First confirm nodes are labeled correctly and that a device plugin is not already running from a previous manual setup, since duplicates cause scheduling conflicts. Add the repo, update it, then install into a dedicated namespace such as gpu-operator, setting driver.enabled and toolkit.enabled based on whether the base OS image already has drivers baked in. For MIG-capable GPUs, set the MIG strategy to single or mixed depending on whether all GPUs on a node share one profile. After installation, verify success by checking that the device plugin and dcgm-exporter pods reach Running state and that a node description shows nvidia.com/gpu in allocatable resources. Version pinning matters here, since operator releases track specific driver and CUDA combinations, and an untested upgrade can break running workloads. Nanobase AI installs and validates the GPU Operator across on-premise, AWS, Azure, and Google Cloud Kubernetes clusters as a standard part of its infrastructure engagements.
Pre-flight checklist before you run Helm
Most failed GPU Operator installs trace back to something that was true about the cluster before Helm ever ran, not a flag chosen during install. Confirm these items first.
| Check | Why it matters |
|---|---|
| Container runtime is containerd or CRI-O with a known config path | The Container Toolkit patches this file; an unusual path breaks auto-configuration |
| No pre-existing nvidia-driver or device plugin DaemonSet | Duplicate device plugins cause GPU double-counting and scheduling errors |
| Node labels reflect real GPU presence | The operator's node selector logic depends on accurate nvidia.com/gpu.present labeling |
| Kernel headers available (self-managed nodes) | The driver container compiles a kernel module on first run and needs headers present |
| Outbound access to NVIDIA's container registry (or a mirrored one) | Air-gapped clusters need images pre-pulled into a private registry first |
Skipping this checklist is the single biggest reason a first GPU Operator install looks broken when the actual problem is an environment mismatch, not the chart itself.
Helm values worth setting explicitly
The default values file assumes a fairly generic bare-metal node. Real clusters usually need at least a few of these overridden.
| Value | When to change it |
|---|---|
driver.enabled | Set false on managed nodes with a pre-installed driver (GKE COS, some EKS AMIs) |
toolkit.enabled | Keep true unless the node image already bundles the container toolkit |
mig.strategy | Set to single or mixed only if MIG-capable GPUs are present |
driver.version | Pin to a tested version rather than tracking the chart's default |
operator.defaultRuntime | Set explicitly if the cluster runs anything other than containerd |
nodeSelector | Scope the operator to GPU-labeled nodes on mixed CPU/GPU clusters |
Overriding these six values in a committed values file, rather than accepting every chart default, is what turns a generic install into one that actually matches your node images and hardware mix.
Step-by-step install and verification
- Add the NVIDIA Helm repository and update it.
- Create a dedicated
gpu-operatornamespace rather than installing intodefault. - Install the chart with your overridden values file, not inline flags, so the configuration is reproducible.
- Watch pod status until
nvidia-driver-daemonset,nvidia-container-toolkit-daemonset,nvidia-device-plugin-daemonset, andnvidia-dcgm-exporterall reach Running. - Run
kubectl describe node <gpu-node>and confirmnvidia.com/gpuappears under Allocatable with the expected count. - Submit a small test pod that requests one GPU and runs
nvidia-smito confirm the full chain works end to end.
Step 5, confirming the allocatable count on the node itself, catches more real install problems than watching pod status alone, since a pod can report Running while the device plugin still advertises zero usable GPUs.
Troubleshooting a stalled install
A validator pod stuck in a non-ready state almost always means one upstream component has not finished, not that the validator itself is broken, so check driver and toolkit pod logs before assuming a validator bug. A device plugin pod in CrashLoopBackOff frequently indicates a duplicate GPU resource advertisement from an old manual install that was never fully removed. If the driver container fails to build its kernel module, check that kernel headers match the running kernel exactly, since a headers package one point release behind the running kernel will silently fail the build. On managed cloud nodes, a driver DaemonSet that keeps restarting almost always means driver.enabled should have been set to false because the node already has one.
Frequently asked questions
Do I need to uninstall an existing manual NVIDIA driver before installing the operator?
Yes, on self-managed nodes. Leaving a manually installed driver in place alongside the operator's driver container causes kernel module conflicts. On managed nodes where the cloud provider installs the driver, you instead disable the operator's driver component rather than removing anything.
How long does a typical GPU Operator install take?
Most of the time is the driver container's first-boot kernel module compile, which can take several minutes per node depending on kernel size and node CPU. Subsequent nodes with a matching kernel and cached image layers come up faster once the first one succeeds.
Can I install the GPU Operator with GitOps tools like Argo CD or Flux?
Yes, and it is the recommended approach for production, since it keeps the Helm values file version-controlled and makes rollbacks a simple revert rather than a manual reinstall scrambled together during an incident, while also giving every cluster a documented, reviewable configuration history.
What is the risk of using helm upgrade without checking release notes?
An operator upgrade can pull a newer driver container that is incompatible with CUDA versions baked into running application images, breaking previously working pods without warning. Always review the compatibility matrix for the target operator version first, and stage the change on a canary node.
How Nanobase AI helps
Nanobase AI installs and validates the NVIDIA GPU Operator across on-premise, AWS, Azure, and Google Cloud Kubernetes clusters as a standard part of its infrastructure engagements, using version-pinned Helm values checked against each customer's driver and CUDA requirements before rollout. See our broader take on Kubernetes GPU Operator vs Slurm if you are still deciding on a scheduler, or explore our platform for the full infrastructure build.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.