Yes, Slurm and Kubernetes can run on the same GPU cluster, either by partitioning nodes between the two schedulers or by running Slurm workloads inside Kubernetes through a bridging project such as Slinky. Slinky, developed jointly by SchedMD and NVIDIA, packages Slurm controller and worker components as Kubernetes-native operators so Slurm jobs are scheduled onto the same node pool that Kubernetes manages, sharing GPU Operator drivers and monitoring rather than duplicating them. The simpler alternative is static partitioning, where a subset of nodes runs a traditional Slurm installation and the rest run Kubernetes, with shared parallel storage such as Lustre or Weka mounted on both sides so datasets and checkpoints stay accessible from either environment. Static partitioning is easier to operate but wastes capacity when one scheduler is idle and the other is queuing jobs. A converged approach with Slinky or a similar bridge lets a single pool of GPUs serve both interactive Kubernetes services and batch Slurm training, improving utilization at the cost of more integration work. Nanobase AI, a Silicon Valley enterprise AI engineering company, has implemented both patterns depending on a customer's existing tooling and team skill set.
Two implementation patterns compared
There are two real ways to make this work, and they trade simplicity against utilization.
| Pattern | How it works | Best for |
|---|---|---|
| Static partitioning | A fixed subset of nodes runs Slurm, the rest run Kubernetes, shared storage mounted on both | Teams wanting minimal integration risk |
| Slinky bridge | Slurm controller and worker components run as Kubernetes operators on the same node pool | Teams wanting one pool of GPUs serving both models |
Static partitioning is the pattern most teams start with because it requires no new software beyond what each scheduler already needs. Slinky, developed jointly by SchedMD and NVIDIA, is the newer approach and removes the capacity-fragmentation problem where one scheduler idles while the other queues jobs, at the cost of a more complex control plane to operate.
Implementing static partitioning step by step
- Decide the node split based on observed or projected demand, not a round number; oversizing the Slurm side is the most common early mistake once inference workloads grow.
- Label and taint Kubernetes-managed nodes so the Kubernetes scheduler never places pods on Slurm-owned hardware, and configure Slurm's node list to exclude Kubernetes-owned hosts.
- Mount the same parallel filesystem, such as Lustre or Weka, on both partitions so datasets and checkpoints are visible regardless of which scheduler produced them.
- Deploy the GPU Operator on the Kubernetes side and a matching driver and CUDA version on the Slurm side manually or via configuration management, keeping both in lockstep.
- Point one Prometheus and Grafana stack at DCGM metrics from both partitions so operators get a single view of fleet health rather than two dashboards.
Getting the node boundary and shared storage mounts right up front avoids the most common failure mode: a training job on the Slurm side that cannot find a checkpoint a Kubernetes-hosted preprocessing job just wrote.
Where Slinky changes the model
Slinky packages the Slurm controller, worker daemons, and accounting components as Kubernetes custom resources, so a Slurm job actually runs as a pod scheduled by Kubernetes underneath, sharing the same GPU Operator-managed driver stack and dcgm-exporter monitoring instead of duplicating them. This lets a single pool of GPUs serve interactive Kubernetes-hosted services and Slurm-submitted batch training jobs, with Kubernetes handling bin-packing across both. The added complexity is real: debugging a stuck job now means understanding both Slurm's queue state and the Kubernetes pod lifecycle underneath it, which requires operators comfortable in both ecosystems rather than specialists in just one.
Common operational pitfalls
Driver and CUDA version drift between the two partitions is the most frequent issue in the static model, since each side is often updated by a different team on a different schedule, and a version mismatch shows up as a job that silently produces wrong results on one partition but not the other. Monitoring gaps are the second common issue: teams that stand up DCGM and Prometheus for the Kubernetes side often forget to instrument the Slurm nodes identically, leaving a blind spot exactly where batch training jobs run longest and are hardest to restart.
Frequently asked questions
Do Slurm and Kubernetes need the same driver version on shared storage?
They need compatible driver and CUDA versions, though not necessarily identical, since CUDA's forward and backward compatibility allows some flexibility. Large version gaps between the two partitions are the most common source of hard-to-debug numerical inconsistencies. Keeping both partitions on the same GPU Operator release, even when driver point-releases differ slightly, is the simplest way to avoid chasing down a mismatch after the fact.
Is Slinky production-ready?
Slinky is an active SchedMD and NVIDIA collaboration and is being adopted by organizations that want a converged scheduler model, but as of 2026 it is newer than either scheduler alone, so teams should pilot it on non-critical capacity first.
Can jobs move between the Slurm and Kubernetes partitions automatically?
Not in the static partitioning model; capacity is fixed once nodes are assigned. A Slinky-based bridge gets closer to dynamic sharing since both scheduling models draw from the same underlying Kubernetes-managed pool. Moving a node between partitions still requires manually re-labeling it and draining any running jobs first, so treat capacity shifts as a planned maintenance action rather than something either scheduler handles on its own.
What is the simplest way to start if we are unsure about a full bridge?
Start with static partitioning and shared storage, since it requires no new control-plane software, and revisit Slinky once utilization data shows the fixed split is actually costing meaningful idle GPU-hours. This also gives the team time to build shared monitoring and driver-version discipline across both partitions before taking on Slinky's added control-plane complexity.
How Nanobase AI helps
Nanobase AI, a Silicon Valley enterprise AI engineering company, has implemented both static partitioning and Slinky-based converged clusters depending on a customer's existing tooling, team skill set, and workload volatility. We size the node split, wire up shared storage and unified monitoring, and validate that both schedulers see consistent driver and CUDA versions before handing the cluster over. See Kubernetes GPU Operator vs Slurm for the underlying trade-offs, or the related question on Kubernetes vs Slurm for GPU workloads.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.