What is Kueue and how does it queue GPU jobs on Kubernetes?

Kueue is a Kubernetes-native job queueing system, built within the Kubernetes SIG ecosystem, that adds the batch scheduling concepts Kubernetes lacks by default, such as quotas, fair sharing, and job admission control, so GPU-hungry batch jobs do not simply flood the cluster and starve each other. Kueue introduces resource flavors to represent different types of GPU hardware, cluster queues to define how much of each flavor a group of teams can consume, and local queues that individual namespaces submit jobs into, admitting a job only once sufficient quota is actually available and holding it pending otherwise rather than letting the default scheduler try and fail repeatedly. This makes Kueue well suited to training and fine-tuning workloads submitted as Kubernetes jobs, where you want ordering and fairness guarantees similar to what Slurm provides natively, layered on top of Kubernetes rather than replacing it. Kueue also supports borrowing, where a team temporarily uses another team's unused quota, and preemption, where higher-priority jobs can reclaim borrowed capacity. It integrates with the standard Kubernetes job API and with frameworks like Kubeflow and KubeRay for machine learning workload types. Nanobase AI, a Silicon Valley enterprise AI engineering company, deploys Kueue when a customer needs Slurm-like batch fairness on a Kubernetes GPU platform rather than running two separate scheduling stacks.

The three objects that make Kueue work

Kueue's model rests on three custom resources that map cleanly onto how most organizations already think about shared GPU capacity: a ResourceFlavor representing a distinct type of hardware, such as H100 versus A100 nodes; a ClusterQueue defining how much of each flavor a group of teams may consume and under what borrowing and preemption rules; and a LocalQueue that a namespace actually submits jobs into, which routes to a ClusterQueue behind the scenes. A job only starts once Kueue's admission controller confirms quota is genuinely available, rather than letting the default Kubernetes scheduler attempt placement and fail repeatedly against a full cluster.

A minimal working setup

apiVersion: kueue.x-k8s.io/v1beta1
kind: ResourceFlavor
metadata:
  name: h100-flavor
spec:
  nodeLabels:
    gpu-type: h100
---
apiVersion: kueue.x-k8s.io/v1beta1
kind: ClusterQueue
metadata:
  name: team-research
spec:
  namespaceSelector: {}
  resourceGroups:
  - coveredResources: ["nvidia.com/gpu"]
    flavors:
    - name: h100-flavor
      resources:
      - name: "nvidia.com/gpu"
        nominalQuota: 8
        borrowingLimit: 4
---
apiVersion: kueue.x-k8s.io/v1beta1
kind: LocalQueue
metadata:
  name: research-queue
  namespace: research
spec:
  clusterQueue: team-research

This grants the research team a nominal quota of 8 GPUs with the ability to borrow up to 4 more from a shared pool when idle, while jobs submitted to research-queue wait in an admitted-or-pending state rather than being scheduled speculatively.

How Kueue compares to the alternatives

Kueue is the right layer when a Kubernetes-based cluster needs Slurm-like batch fairness without adopting Slurm itself, and when the fractional-sharing needs are met by MIG or time-slicing configured through the GPU Operator rather than requiring Run:ai's finer-grained fractioning.

Where Kueue fits with training frameworks

Kueue integrates with the standard Kubernetes Job API directly, and with Kubeflow's training operator and KubeRay for framework-native distributed jobs, meaning a PyTorchJob or RayJob custom resource can be queued and admitted the same way as a plain batch Job. This matters for gang scheduling: a multi-pod distributed training job should not have half its pods start while the rest wait indefinitely for capacity, and Kueue's workload abstraction treats the whole job as one admission unit rather than admitting pods individually.

Borrowing and preemption round out the picture for shared clusters. A team that temporarily needs more than its nominal quota can borrow from another team's unused allocation, and Kueue reclaims that borrowed capacity through preemption once the lending team's own jobs need it back, which avoids the common failure mode where reserved-but-idle GPUs sit unused while a queue outside that reservation backs up.

Frequently asked questions

Does Kueue replace the Kubernetes scheduler?

No. Kueue works alongside the default scheduler by controlling when a workload becomes eligible for scheduling, through admission, rather than replacing how pods get placed onto nodes once admitted. The default scheduler, or a custom one, still handles actual pod placement.

Can Kueue schedule fractional GPUs like Run:ai?

Kueue itself schedules at the resource-quantity level and relies on the underlying GPU Operator's MIG or time-slicing configuration to expose fractional GPU units to Kubernetes. It does not add its own dynamic GPU fractioning layer the way Run:ai's scheduler does.

Is Kueue mature enough for production GPU clusters?

Yes. Kueue is a Kubernetes SIG project with active development and is used in production GPU batch scheduling at meaningful scale as of 2026, though teams evaluating it should still test their specific job types, particularly multi-pod gang-scheduled jobs, before full rollout.

How Nanobase AI helps

Nanobase AI deploys Kueue for customers who need fair, quota-based GPU batch scheduling on Kubernetes without taking on a separate Slurm stack or a commercial scheduler's licensing cost, configuring ClusterQueues and borrowing policy around each team's actual usage pattern. We validate the setup against real multi-pod training jobs before handing it over, not just synthetic test jobs. See how this fits into a broader platform on our solutions page.

Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.