What is NVLink vs InfiniBand and when do you need each?

NVLink connects GPUs within and across a small number of nodes at very high bandwidth for tightly coupled communication, while InfiniBand connects entire nodes across a data center-scale fabric, so the two solve different problems rather than competing directly. NVLink 4 on H100 provides 900 GB/s of GPU-to-GPU bandwidth within a server, and NVLink 5 on Blackwell systems extends into NVSwitch-based domains spanning multiple physical nodes, such as a 72-GPU rack-scale system, making it effectively intra-domain rather than strictly intra-node. InfiniBand, by contrast, links separate NVLink domains or separate servers together, carrying gradient synchronization traffic between nodes during distributed training. You need strong NVLink when a model's tensor-parallel shards must communicate constantly within a single server, and you need InfiniBand once training spans more nodes than one NVLink domain covers, since data-parallel or pipeline-parallel gradient exchange must then cross the node boundary. Getting this wrong, for example relying on standard Ethernet for tensor-parallel traffic, causes severe throughput collapse. Nanobase AI plans interconnect topology, choosing NVLink domain size and InfiniBand fabric scale together, as part of every multi-node cluster design.

Scope and bandwidth compared

NVLink and InfiniBand solve different problems rather than competing for the same job, and treating them as interchangeable is where multi-node training designs go wrong.

Mapping parallelism strategy to the right interconnect

1. Tensor-parallel shards that must exchange activations on every layer need the lowest possible latency, which means they must stay within a single NVLink domain rather than crossing to InfiniBand.
2. Pipeline-parallel stages exchange activations less frequently, once per micro-batch boundary, tolerating the higher latency of an InfiniBand hop between nodes reasonably well.
3. Data-parallel gradient synchronization happens once per training step across all replicas and is the traffic pattern InfiniBand was built to carry efficiently at scale.
4. Expert-parallel routing in mixture-of-experts models generates bursty, unpredictable communication patterns that benefit from staying within an NVLink domain wherever the expert count allows it.

Getting this mapping wrong, for example placing tensor-parallel shards across a standard Ethernet link instead of within an NVLink domain, causes severe throughput collapse rather than a modest slowdown, since every layer's forward and backward pass now waits on a comparatively slow network hop.

What happens when NVLink domains grow past one node

NVLink 5 on Blackwell-generation systems, through NVSwitch-based rack-scale designs such as a 72-GPU domain, extends what used to be a strictly intra-node concept into a multi-node domain that still behaves like one tightly coupled interconnect from a software perspective. This changes cluster planning meaningfully: a workload that previously needed InfiniBand the moment it crossed a physical server boundary can now stay on NVLink across several physical nodes as long as they belong to the same NVSwitch domain, only falling back to InfiniBand once communication needs to cross domain boundaries entirely.

Planning domain size and fabric scale together

Interconnect topology is not two independent decisions; NVLink domain size determines how much tensor and expert parallelism a single domain can absorb before pipeline or data parallelism, and therefore InfiniBand, has to take over. A cluster architect sizing a training job needs to know both numbers together: how many GPUs share NVLink, and how many InfiniBand hops separate the rest, since the parallelism strategy has to be designed against that exact topology rather than assumed generically.

Frequently asked questions

Do I need InfiniBand if all my GPUs fit in one NVLink domain?

Only if you plan to scale beyond that domain later or need external connectivity for storage and orchestration traffic. A workload that permanently fits within a single NVLink domain, including newer rack-scale domains, may never need InfiniBand for its actual training communication.

Can NVLink and InfiniBand carry the same traffic simultaneously?

They typically don't overlap in role; NCCL automatically routes intra-domain traffic over NVLink and inter-domain traffic over InfiniBand based on detected topology, so a well-configured job uses both without manual traffic splitting. This topology detection happens automatically at initialization, so operators rarely need to configure NCCL's routing choices by hand unless diagnosing an unexpected performance regression.

Is NVLink 5's larger domain size available on all Blackwell systems?

No, it depends on the specific system design; rack-scale NVSwitch domains require purpose-built chassis and switch infrastructure, so a standard 8-GPU Blackwell server does not automatically gain the larger multi-node NVLink domain without that infrastructure. Confirm domain size directly with the system vendor during procurement, since chassis-level NVLink topology varies meaningfully across otherwise similar Blackwell server offerings.

What is the practical sign that tensor parallelism crossed an interconnect boundary it shouldn't have?

A sudden, disproportionate drop in step throughput when scaling a tensor-parallel group beyond a certain GPU count is the classic symptom, usually traced back to that group spanning more than one NVLink domain or crossing to a standard network link.

How Nanobase AI helps

Nanobase AI, a Silicon Valley enterprise AI engineering company, plans interconnect topology, choosing NVLink domain size and InfiniBand fabric scale together, as part of every multi-node cluster design, then maps each customer's chosen parallelism strategy against that topology before training starts. For interconnect basics, see do you need InfiniBand for a GPU cluster, and for model sizing see how many GPUs for 70B and 405B models.

Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.