MIG vs time-slicing: how do we share GPUs between users?

MIG gives hard, hardware-enforced isolation between users sharing a GPU, while time-slicing gives soft sharing where the GPU scheduler rapidly switches between processes without memory or fault isolation, so the right choice depends on whether workloads need predictable performance or just access to idle capacity. MIG suits production inference or multi-tenant environments where one user's workload must never affect another's memory or throughput, since each instance has dedicated memory and cannot be starved by a noisy neighbor. Time-slicing is simpler to configure, requires no hardware partitioning, works on GPUs that do not support MIG such as A10 or L40S, and suits development environments, CI pipelines, or bursty low-priority jobs where occasional contention is acceptable. The trade-off is that time-sliced workloads compete for the same memory pool, so an out-of-memory error in one job can affect others, and total throughput is not guaranteed. Many clusters use both, reserving MIG-partitioned H100s for tenant-isolated inference and time-slicing on older GPUs for shared development. Nanobase AI, a Silicon Valley enterprise AI engineering company, configures GPU Operator sharing policies to match each customer's isolation and utilization requirements rather than defaulting to one approach.

Side-by-side comparison

The decision is really about whether a noisy neighbor is acceptable, and for anything with a latency SLA the answer is almost always no, which points straight to MIG wherever the hardware supports it.

GPUs that need time-slicing because they cannot do MIG

MIG is limited to Ampere-generation and newer data center GPUs: A100, H100, H200, and Blackwell-generation parts. Widely deployed inference GPUs such as L40S, L4, and A10 do not support MIG at all, so time-slicing, or the closely related Multi-Process Service, is the only software-based sharing option available on that hardware. This matters when planning a cluster that mixes GPU generations, since a fleet with older or smaller GPUs cannot simply adopt a MIG-only sharing policy across the board.

Configuring both in the same GPU Operator deployment

The GPU Operator supports MIG and time-slicing as independent ConfigMap settings, and a cluster commonly runs both at once on different node pools. MIG-capable H100 nodes serving tenant-isolated production inference get a MIG profile applied, while older L40S or A10 nodes reserved for development and CI get a time-slicing replica count instead, for example splitting one physical GPU into four logical replicas for the scheduler. Both configurations coexist under the same operator install, referenced by different node labels, so a single cluster does not need two separate GPU management stacks to support both sharing models.

A third option: Multi-Process Service

NVIDIA's Multi-Process Service, or MPS, sits between the two: it lets multiple processes share a GPU's compute concurrently rather than time-slicing between them sequentially, improving throughput for many small concurrent kernels, but it still shares one memory space and offers no fault isolation between processes. MPS is worth evaluating specifically when small workloads are compute-bound and would otherwise sit idle waiting their turn under pure time-slicing, though it adds a layer of operational complexity that many teams skip unless they have already measured a real throughput problem with plain time-slicing.

Frequently asked questions

Can MIG and time-slicing run on the same physical GPU at once?

No, they are mutually exclusive per GPU. A GPU is either partitioned into MIG instances or shared via time-slicing, not both simultaneously, though a node with multiple GPUs can mix approaches across its cards using the operator's mixed configuration options.

Does time-slicing guarantee each job a fair share of GPU time?

Not strictly. Time-slicing gives access to idle capacity but does not enforce fairness or throughput guarantees the way a CPU scheduler's fair-share policy might, so one aggressive job can still dominate GPU cycles relative to others sharing the card, and the GPU Operator applies no priority weighting between replicas by default.

Is time-slicing safe for production inference?

Generally not for latency-sensitive SLAs, since a memory spike or long-running kernel from one tenant can degrade or crash another sharing the same physical memory pool. MIG is the safer choice wherever the hardware supports it and isolation matters. On hardware that cannot do MIG at all, such as L40S or A10, time-slicing with careful memory-footprint sizing per replica is the fallback rather than a genuinely safe default.

How many time-slicing replicas should I configure per GPU?

There is no universal number; it depends on workload memory footprint and acceptable contention. Start conservative, around two to four replicas, and adjust based on observed queuing and out-of-memory events rather than maximizing replica count upfront. Revisit the number whenever workload memory footprints change materially, since a replica count sized for one model generation can silently become too aggressive for a larger successor.

How Nanobase AI helps

Nanobase AI, a Silicon Valley enterprise AI engineering company, configures GPU Operator sharing policies to match each customer's isolation and utilization requirements, mixing MIG and time-slicing across node pools rather than defaulting to one approach cluster-wide. For MIG-specific sizing, see NVIDIA MIG profiles and sizing.

Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.