The H100 SXM and H100 PCIe are the same Hopper GPU packaged differently, with SXM built for the highest performance multi GPU systems and PCIe built for flexibility in standard servers. The SXM5 module runs at up to 700 watts, delivers the full 80 GB of HBM3 at 3.35 TB/s, and connects through fourth generation NVLink at 900 GB/s across an 8 GPU baseboard found in systems like DGX H100 and HGX H100. The PCIe card is rated around 300 to 350 watts, uses a standard PCIe Gen5 x16 slot, offers somewhat lower memory bandwidth, and supports NVLink bridging only between pairs of adjacent cards rather than a full 8 GPU mesh. PCIe cards are easier to add to existing rack servers, require less specialized power and cooling infrastructure, and typically cost less per GPU, but they underperform SXM systems on workloads that depend heavily on GPU to GPU communication such as large model training or tensor parallel inference across many GPUs. Choosing between them depends on whether the workload needs full NVLink bandwidth or fits comfortably on one to four GPUs. Nanobase AI matches SXM or PCIe H100 configurations to each client's actual parallelism requirements.
Same die, different deployment target
| Spec | H100 SXM5 | H100 PCIe |
|---|---|---|
| TDP | up to 700 W | ~300–350 W |
| Memory bandwidth | 3.35 TB/s | somewhat lower |
| NVLink | 4th gen, 900 GB/s, full 8-GPU mesh via NVSwitch | Bridge between adjacent pairs only |
| Interface | Custom SXM5 socket on baseboard | Standard PCIe Gen5 x16 slot |
| Typical system | HGX H100, DGX H100 (8-GPU baseboard) | Standard 1U–4U rack servers |
| Deployment flexibility | Requires SXM-qualified chassis | Drops into most existing rack servers |
The single most consequential difference is NVLink topology: SXM5 GPUs sit on a baseboard connected through NVSwitch so all eight GPUs communicate at full 900 GB/s bandwidth with each other, while PCIe cards can only bridge NVLink between two adjacent cards, leaving anything beyond a pair to fall back to much slower PCIe.
When PCIe is the right call
PCIe H100s make sense when the deployment needs one to four GPUs rather than a full eight, when the organization wants to add GPU capacity to an existing standard rack server rather than deploying a purpose-built 8-GPU chassis, or when power and cooling budgets are tighter and the lower 300–350 W TDP meaningfully simplifies facility requirements. They are also typically less expensive per GPU and easier to source through general server integrators rather than requiring an HGX or DGX-qualified system.
When SXM is worth the added complexity
SXM is the better choice whenever the workload depends on GPU-to-GPU communication across more than two GPUs, which covers most tensor-parallel or pipeline-parallel training, and most high-concurrency inference serving of models larger than what fits comfortably on one or two GPUs. Because all eight GPUs sit in one NVLink domain, sharding a large model across the full baseboard avoids the PCIe bottleneck that would otherwise stall parallel workloads waiting on cross-GPU data transfer.
Decision checklist
- Count how many GPUs a single inference or training job actually needs to span. One or two: PCIe is likely sufficient. Four or more with heavy cross-GPU traffic: SXM.
- Check whether the target workload is tensor-parallel (heavy, constant cross-GPU communication) or data-parallel (GPUs mostly work independently). Data-parallel workloads tolerate PCIe's limitations better.
- Confirm facility power and cooling budget against the roughly 2x power difference between PCIe and SXM at the 8-GPU level.
- Weigh procurement flexibility: PCIe cards fit into a much broader range of existing server chassis than SXM's baseboard requirement.
- If unsure, benchmark the actual model's parallelism pattern rather than assuming SXM is always necessary — many single-GPU or two-GPU inference deployments never need the full NVLink mesh.
See also how power draw differs between SXM and PCIe builds and why memory bandwidth drives inference performance.
Frequently asked questions
Can H100 PCIe cards use NVLink at all?
Yes, but only in pairs through an NVLink bridge connecting two adjacent cards, not across the full set of GPUs in a server the way SXM's NVSwitch-based baseboard does.
Is H100 PCIe meaningfully slower for single-GPU inference?
For workloads that fit on a single GPU and do not need cross-GPU communication, the difference is smaller and mostly comes down to memory bandwidth, since NVLink topology is irrelevant when only one GPU is involved.
Does SXM always require a DGX or HGX system?
SXM modules require an SXM-qualified baseboard, which is what DGX and HGX systems provide, so yes — SXM cannot be added to a generic PCIe rack server the way a PCIe card can.
Which form factor is better for fine-tuning large models?
SXM is generally preferable for fine-tuning models that require sharding across more than two GPUs, since gradient synchronization traffic benefits significantly from the full NVLink mesh rather than PCIe's pairwise bridging.
How Nanobase AI helps
Nanobase AI, an enterprise AI engineering company with engineering headquarters in Silicon Valley, matches SXM or PCIe H100 configurations to each client's actual parallelism requirements rather than defaulting to the highest-spec option, keeping both cost and facility complexity aligned with real workload needs. Learn more about our GPU sizing and deployment work.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.