Azure offers H100 GPUs through the ND H100 v5 series and H200 GPUs through the ND H200 v5 series, both built for large scale AI training and inference. ND H100 v5 virtual machines pair eight NVIDIA H100 GPUs, each with 80 GB of HBM3 at 3.35 TB/s of bandwidth, with InfiniBand interconnects between nodes for distributed training and multi GPU inference. ND H200 v5 instances keep the same eight GPU configuration but upgrade to H200 silicon, raising memory to 141 GB of HBM3e per GPU at about 4.8 TB/s, which mainly benefits memory bound inference workloads such as long context serving and larger batch sizes. Azure also offers NC series virtual machines with smaller GPU counts for lighter workloads, and newer ND series generations built around NVIDIA Blackwell are rolling out for the largest training jobs. Quota for ND H100 v5 and ND H200 v5 typically requires a support request with an approved use case, and availability varies significantly by region. As an NVIDIA Inception Program member, Nanobase AI helps enterprises pick the right ND series size and provision Azure GPU quota efficiently.
Choosing between ND, NC and the newer Blackwell-based series
Azure's GPU virtual machine lineup spans more than just ND H100 v5 and ND H200 v5, and picking the wrong tier is a common early mistake. NC-series virtual machines target lighter inference or development workloads with fewer GPUs per node and lower cost, while ND H100 v5 and ND H200 v5 are built for multi-node training and high-concurrency inference with InfiniBand between nodes. Newer ND series generations built on NVIDIA Blackwell are rolling out for the largest training jobs as of 2026 but carry the same early-adoption software risk as any first-generation instance family. A team serving a single 70B model with moderate traffic rarely needs full ND H100 v5 capacity; that scale is where NC-series or a single ND node without multi-node InfiniBand tuning already suffices.
The quota request process, step by step
- Confirm the target Azure region actually stocks ND H100 v5 or ND H200 v5 capacity, since availability is uneven as of 2026.
- Open a support request specifically for GPU core quota on the target VM family, not a generic compute quota request.
- Include a concrete workload description: model, expected GPU count, training or inference, and expected duration.
- Loop in the Azure account team early if the request is large; self-service quota requests for these families are frequently escalated for review regardless of size.
- Evaluate Azure's reserved capacity options if the start date is known, since reservations reduce both cost and availability risk compared to pure pay-as-you-go.
- Validate InfiniBand connectivity with a small multi-node test before committing to a full-scale deployment.
A specific, workload-grounded request that names the exact VM family, region and duration moves through Azure's review process far faster than a generic compute quota ask.
InfiniBand and multi-node inference: what changes at scale
A single ND H100 v5 or ND H200 v5 node already delivers eight GPUs with NVLink between them, which covers most 70B-class inference deployments in FP8 without ever touching cross-node networking. The InfiniBand fabric between nodes matters once a workload needs tensor or pipeline parallelism spanning more than one node's GPUs, which is more common in large-scale training or serving models well beyond 70B parameters. Getting multi-node performance right requires proximity placement groups and accelerated networking configured correctly at deployment time, since a virtual machine deployed without these settings will run but silently underperform on any workload that depends on cross-node communication.
H100 versus H200: what the memory upgrade actually buys
ND H100 v5 and ND H200 v5 share the same eight-GPU, InfiniBand-connected architecture, so the practical difference comes down to per-GPU memory: 80 GB of HBM3 at 3.35 TB/s versus 141 GB of HBM3e at about 4.8 TB/s. That extra memory mainly helps workloads that are memory-bound rather than compute-bound, meaning longer context windows, larger batch sizes, and serving bigger models without sharding across more GPUs than necessary. A team running short-context, moderate-concurrency inference on a 70B model often sees little practical difference between the two, while a team serving long documents or high-concurrency chat sees the H200's headroom directly in how many simultaneous sessions one node supports. Choosing H200 over H100 should follow directly from a measured memory bottleneck, not from assuming the newer chip is always the better default.
Frequently asked questions
How long does Azure GPU quota approval typically take?
It ranges from a few business days for smaller, well-justified requests to several weeks for large ND H100 v5 or ND H200 v5 allocations, particularly during periods of tight regional supply. Engaging the Azure account team ahead of the actual need generally shortens this.
Do I need InfiniBand for a single-node inference deployment?
No. InfiniBand matters for multi-node communication; a single ND H100 v5 or ND H200 v5 node already uses NVLink internally between its eight GPUs, which covers most single-node inference workloads without any InfiniBand configuration.
Is ND H200 v5 worth the upgrade over ND H100 v5 for inference?
It depends on whether the workload is memory-bound. Long-context serving, large batch sizes, or models near the edge of what fits in 80 GB per GPU benefit meaningfully; short-context, moderate-traffic deployments often do not need the extra memory.
Which Azure regions carry ND H100 v5 or ND H200 v5 capacity?
Availability changes over time and varies by family; it should be checked directly in the Azure portal or with an account representative rather than assumed from a general region list, especially for newer H200 or Blackwell-based series.
How Nanobase AI helps
As an accepted member of the NVIDIA Inception Program, Nanobase AI helps enterprises choose between NC-series, ND H100 v5, and ND H200 v5 based on actual model size and traffic rather than defaulting to the largest available VM, then manages the Azure quota request and InfiniBand validation needed to make multi-node deployments perform as expected. See our on-premise and cloud GPU deployment guidance and solutions for the broader infrastructure approach.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.