What is NVFP4 and can Blackwell run 4-bit models faster?

NVFP4 is NVIDIA's native 4-bit floating-point format introduced for the Blackwell architecture, using fine-grained, per-block scaling rather than a single scale factor across an entire tensor, which generally preserves more accuracy than older, coarser INT4 quantization schemes at the same bit width. Blackwell GPUs like the B200 include hardware-accelerated tensor cores specifically for FP4 formats, so a model quantized to NVFP4 can run meaningfully faster and use roughly half the memory of FP8 while retaining much of FP8's accuracy profile, assuming the serving framework and model support it. As of 2026, support for NVFP4 across serving engines such as TensorRT-LLM and vLLM is still maturing compared to the well-established FP8 and INT4 paths on Hopper hardware, so the realized speedup depends heavily on whether a given model has a validated NVFP4 checkpoint and kernel support. For teams already committed to Blackwell hardware, NVFP4 is worth evaluating as a way to fit larger models or higher concurrency into the same GPU footprint. For Hopper-based H100 or H200 deployments, FP8 remains the more mature default. Nanobase AI, an NVIDIA Inception Program member with early access to Blackwell-class hardware guidance, tracks NVFP4 support across serving frameworks and validates it on customer models before recommending it for production.

NVFP4 compared to FP8 and INT4

NVFP4 sits between FP8 and traditional INT4 in the trade-off between memory, speed and accuracy, and the comparison is clearest laid out side by side.

Key takeaway: NVFP4 targets the same memory savings as traditional INT4 but with meaningfully better accuracy, at the cost of needing Blackwell hardware and newer software support.

Why per-block scaling changes the accuracy story

Traditional INT4 methods like AWQ and GPTQ typically apply one or a small number of scale factors across a large portion of a weight tensor, which works reasonably well but loses precision when values within that portion vary widely. NVFP4 uses much finer, per-block scaling, computing separate scale factors for small groups of values, which tracks the actual distribution of weights far more closely and is a major reason it preserves more accuracy than older, coarser 4-bit schemes at the same 4-bit storage cost. This finer scaling requires more computation during quantization and more sophisticated hardware support to execute efficiently at inference time, which is exactly what Blackwell's dedicated FP4 tensor cores are built for.

Key takeaway: NVFP4's accuracy advantage over older INT4 methods comes from finer-grained scaling, not from a fundamentally different number of bits.

What to check before adopting NVFP4 in 2026

1. Confirm your target model has a validated NVFP4 checkpoint or a supported path to produce one, since not every model has this readily available yet.
2. Verify the serving engine you plan to use, TensorRT-LLM or vLLM, has stable NVFP4 kernel support for your model's architecture, since support is still rolling out unevenly across engines.
3. Confirm the hardware is Blackwell-based (B200 or RTX PRO 6000-class); NVFP4's native acceleration does not extend to Hopper or earlier generations.
4. Run your own accuracy comparison against FP8 on representative prompts, since NVFP4's accuracy advantage is over traditional INT4, not necessarily over FP8.
5. Weigh the realized speed and memory benefit against the ecosystem maturity risk of adopting a newer format before committing a production deployment to it.

Key takeaway: NVFP4 is worth piloting on Blackwell hardware in 2026, with the understanding that support across models and engines is not yet as uniform as FP8's.

What NVFP4 means in concrete GPU count

For a 70B model, NVFP4 targets roughly the same memory footprint as traditional INT4, about 38 GB, which fits comfortably on a single B200 with substantial headroom, versus needing an H100 or H200 at FP8's roughly 70 GB. The practical gain on Blackwell hardware is not a smaller GPU count than FP8 would already achieve at this model size, since both fit on one GPU; it is the combination of INT4-class memory savings with accuracy closer to FP8, which matters more as model size grows toward the point where the memory difference between formats starts to change GPU count rather than just headroom.

Key takeaway: NVFP4's real advantage shows up most clearly at larger model sizes, where the memory gap between FP8 and 4-bit precision starts to determine GPU count rather than just headroom. For how NVFP4 fits alongside the other formats, see FP16 vs BF16 vs FP8 vs INT4.

Frequently asked questions

Does NVFP4 replace FP8 as the recommended default?

Not yet as of 2026. FP8 remains the more broadly validated and supported default across both Hopper and Blackwell hardware. NVFP4 is the format to evaluate specifically when Blackwell is the target hardware and the workload needs the extra memory and speed headroom that FP8 does not provide.

Can NVFP4 models run on H100 or H200?

No, not with native acceleration. NVFP4 depends on Blackwell's dedicated FP4 tensor cores; running an NVFP4-quantized model on Hopper hardware would require falling back to unaccelerated execution, which defeats the purpose of the format.

How much faster is NVFP4 than FP8 on a B200?

NVFP4 roughly halves the memory footprint of FP8 and Blackwell's FP4 tensor cores offer substantially higher peak throughput than its FP8 tensor cores for the same operation, though the realized end-to-end speedup for a given model depends on how much of the workload is compute-bound versus memory-bound.

Is NVFP4 only useful for inference, or does it help training too?

As of 2026, NVFP4's primary validated use case is inference; some experimental work applies FP4-class formats to parts of training, but this is far less mature than FP4 inference and not yet a standard recommendation.

How Nanobase AI helps

Nanobase AI, an accepted member of the NVIDIA Inception Program with early access to Blackwell-class hardware guidance, tracks NVFP4 support across serving frameworks and validates it against FP8 on customer models before recommending it for production. We help determine whether the ecosystem maturity for your specific model and engine combination justifies adopting NVFP4 today or waiting. See our H100 vs H200 vs B200 guide for the broader hardware picture.

Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.