An 8x H100 SXM server typically draws between about 8 and 11 kilowatts under full load, with the eight GPUs alone accounting for roughly 5.6 kilowatts at their 700 watt SXM5 TDP, and the remainder coming from dual CPUs, memory, NVMe storage, NICs, and fan or pump power. Reference systems such as the NVIDIA DGX H100 are commonly rated around 10.2 kilowatts of maximum power draw, so facility planning should budget for at least that figure per node plus headroom for power supply inefficiency and transient spikes during training. PCIe based H100 configurations draw meaningfully less, since each PCIe card is rated around 300 to 350 watts rather than 700 watts, which lowers total server power to roughly 4 to 6 kilowatts for an 8 GPU build. Actual consumption varies with workload, since inference at low batch sizes rarely hits sustained peak power while large training runs often do. Data center operators should size power distribution units, circuit breakers, and cooling capacity to the sustained peak rather than the idle average. Nanobase AI, a Silicon Valley enterprise AI engineering company, performs detailed power and cooling assessments before any H100 installation to avoid under provisioned electrical infrastructure.
Where the watts actually go
| Component | Approximate draw (8x H100 SXM) |
|---|---|
| 8x H100 SXM5 GPUs (700 W each) | ~5.6 kW |
| Dual server-class CPUs | 0.5–1.0 kW |
| System memory (1–2 TB) | 0.1–0.3 kW |
| NVMe storage | 0.1–0.2 kW |
| NICs / InfiniBand adapters | 0.1–0.3 kW |
| Fans, PSU losses, misc | 0.5–1.5 kW |
| Total (typical) | ~8–11 kW |
The GPUs are the majority of the load but not the whole story: real facility planning has to account for roughly 2.5 to 5 kW of non-GPU draw on top of the 5.6 kW the eight GPUs consume at their rated TDP. Reference systems like the DGX H100 are commonly rated around 10.2 kW of maximum power draw, which is a reasonable per-node planning figure that already bakes in this overhead.
SXM versus PCIe changes the math significantly
| Configuration | Per-GPU TDP | 8-GPU total (GPUs only) | Typical full-server total |
|---|---|---|---|
| H100 SXM5 | up to 700 W | ~5.6 kW | ~8–11 kW |
| H100 PCIe | ~300–350 W | ~2.4–2.8 kW | ~4–6 kW |
A PCIe-based 8-GPU build draws roughly half the power of an SXM build, which matters directly for facilities with limited circuit capacity, but comes with the tradeoff of lower memory bandwidth and NVLink bridging only between adjacent GPU pairs rather than a full NVLink mesh — see the full SXM vs PCIe comparison for the tradeoffs beyond power.
Sizing facility power correctly
- Use the sustained peak figure (roughly 10.2 kW for an SXM reference design), not the idle or average draw, when sizing circuits and PDUs.
- Add headroom for power supply inefficiency and transient spikes, particularly during the start of large training jobs when GPUs ramp from idle to full load quickly.
- Confirm the data center's power distribution units and upstream circuits are rated for sustained draw at this density, not just momentary peaks.
- Plan redundant power feeds (A/B power) per node for production workloads, which roughly doubles the PDU capacity requirement per rack even though average draw does not change.
- Multiply per-node figures by planned node count early, since an 8-node H100 SXM cluster can require 80–90 kW of sustained facility power before adding networking and storage infrastructure.
Inference load rarely hits sustained peak
Actual consumption varies meaningfully with workload. Low-batch inference serving rarely pushes GPUs to their full 700 W TDP continuously, since utilization dips between requests, while large training runs with continuous back-to-back batches tend to sit much closer to sustained peak for hours at a time. Facility planning should still budget for the peak figure, since under-provisioning based on typical inference utilization risks tripping breakers the first time a training job or traffic spike pushes the cluster to full load.
Frequently asked questions
Does an H100 server ever draw more than its rated TDP?
Momentary transient spikes above steady-state draw are possible during load ramp-up, which is why facility planning includes headroom above the nameplate figure rather than sizing circuits to the exact rated wattage.
How much less power does H100 PCIe use than SXM?
Roughly half at the GPU level: PCIe cards are rated around 300–350 W versus 700 W for SXM5, which brings full 8-GPU server draw down from roughly 8–11 kW to about 4–6 kW.
Is 10.2 kW a safe planning number for any 8x H100 SXM server?
It is a reasonable reference point based on DGX H100, but always confirm the specific OEM server's rated maximum power draw, since CPU choice, storage configuration, and networking cards can shift the total by a kilowatt or more.
Does idle power matter for facility planning?
Idle power matters for ongoing electricity cost estimates but not for circuit and PDU sizing, which must be based on sustained peak draw to avoid overloads during training runs or traffic spikes.
How Nanobase AI helps
Nanobase AI, an enterprise AI engineering company with engineering headquarters in Silicon Valley, performs detailed power and cooling assessments before any H100 installation, sizing circuits and PDUs to sustained peak draw rather than average utilization to avoid under-provisioned electrical infrastructure. See how we approach GPU infrastructure planning.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.