Running the full DeepSeek R1 model, a 671 billion parameter mixture-of-experts model, requires substantial GPU capacity even at reduced precision, since its native FP8 weights alone total around six hundred seventy to seven hundred gigabytes, meaning a realistic deployment needs at least eight H200 GPUs in one node or a multi-node H100 cluster with InfiniBand for the cross-node communication that both tensor and expert parallelism require. Both vLLM and SGLang support DeepSeek R1, with SGLang notably shipping strong day-one performance and specific optimizations for its multi-head latent attention and expert parallelism, while vLLM added comparable support shortly after; benchmarking both on your own cluster is worthwhile since relative performance shifts with each release. For teams without that much GPU capacity, the distilled DeepSeek R1 variants built on Qwen and Llama backbones, ranging from about 1.5B to 70B parameters, run comfortably on one to a few GPUs and retain much of the reasoning behavior at a fraction of the resource cost. A practical consideration specific to R1 is its verbose reasoning output, or thinking tokens, which increases total generated tokens per response and should be factored into throughput and cost planning. Nanobase AI has sized and deployed both full and distilled DeepSeek R1 clusters for enterprise customers.
Pick the deployment tier before touching a config file
The first decision is not vLLM versus SGLang, it is which DeepSeek R1 you are actually running. The naming is a frequent source of confusion: the distilled checkpoints (1.5B through 70B, built on Qwen and Llama backbones) are dense models that behave like any other dense model at that size, while the full 671B mixture-of-experts model is a categorically different infrastructure problem.
| Variant | Parameters | Typical fit | Node count |
|---|---|---|---|
| R1-Distill-Qwen-1.5B/7B | 1.5B–7B | Single GPU, FP16 or FP8 | 1 |
| R1-Distill-Llama-8B / Qwen-14B/32B | 8B–32B | Single GPU (A100/H100) or two with tensor parallelism | 1 |
| R1-Distill-Llama-70B | 70B | 2–4 GPUs at FP8, tensor-parallel | 1 |
| Full DeepSeek R1 (MoE) | 671B (37B active) | 8x H200 in one node, or multi-node H100 with InfiniBand | 1–2+ |
The right starting question is "which variant fits my hardware and latency target," not "which engine is faster," since the answer to the second question depends entirely on the first.
Why the full model needs multi-node thought, not just more GPUs
DeepSeek R1's 671B total parameters only activate about 37B per token thanks to its mixture-of-experts routing, but every expert still needs to live in GPU memory somewhere, so the full weight footprint (roughly 670–700 GB at FP8) has to be sharded across devices regardless of how few experts fire on any given token. This is why a realistic full-model deployment starts at eight H200 GPUs in a single NVLink-connected node, or a multi-node H100 cluster where InfiniBand carries the tensor-parallel and expert-parallel traffic between nodes.
Multi-node serving introduces failure modes that single-node deployments never see: a slow or flapping InfiniBand link degrades the whole cluster's throughput, not just one GPU's, and node startup order matters because the model shards need to find each other before the first request can be served. Both vLLM and SGLang use Ray for multi-node orchestration, so getting Ray's cluster networking right (correct head node address, matching CUDA and NCCL versions across nodes) is usually the actual blocker, not the inference engine configuration itself.
Full DeepSeek R1 is a multi-node systems problem first and an inference-tuning problem second.
The KV cache math looks different because of MLA
DeepSeek R1 uses multi-head latent attention (MLA) instead of standard multi-head or grouped-query attention, which compresses the key-value representations into a much smaller latent space before caching them. In practice this means R1's KV cache per token consumes noticeably less memory than a standard-attention model of comparable size would, which is one reason SGLang, whose team contributed early MLA-specific kernel optimizations, showed strong performance on R1 from its earliest release. vLLM has since closed most of that gap with its own MLA-aware kernels, but the two engines' relative performance on this specific architecture shifts with nearly every release, so a real benchmark on your own cluster and your own request shapes is worth more than any external claim, including this one.
MLA is the specific architectural reason R1 fits more concurrency per GPU than its raw parameter count would suggest.
Reasoning tokens change your capacity planning
R1 produces visible chain-of-thought reasoning before its final answer, and that reasoning text counts as generated tokens like any other output. A prompt that would produce a 200-token answer from a non-reasoning model can easily produce several times that once reasoning tokens are included, which inflates both latency and cost per request in ways that a token-per-second benchmark alone will not reveal. Two practical mitigations: set a max output token budget appropriate to the task rather than the model default, and consider speculative decoding with a small draft model for latency-sensitive interactive use, since the token-by-token generation pattern of decode-heavy reasoning output is exactly the workload speculative decoding helps most.
Budget for reasoning-token overhead explicitly, since it changes both latency and cost math more than the base model size does.
Frequently asked questions
Can I run DeepSeek R1 on a single H100 node?
Only a distilled variant, not the full 671B model. A single 8x H100 node (640 GB total HBM) is tight for the full model even at FP8 once activation memory and KV cache are added; most production full-R1 deployments use H200 nodes or span multiple H100 nodes with InfiniBand.
Does SGLang still outperform vLLM for R1 as of 2026?
Not reliably as a blanket statement. SGLang had an early lead from MLA-specific optimizations, but vLLM has added comparable support since, and relative performance now depends on your exact hardware, batch shape, and engine version. Benchmark both on your own traffic rather than relying on either project's release-day claims.
How much GPU memory do the distilled variants need?
They follow standard dense-model sizing: the 70B distill needs roughly the same memory as any other 70B model (about 140 GB FP16, 70 GB FP8, or under 40 GB at INT4 for weights alone), while the 7B and 8B distills run comfortably on a single 24 GB GPU at FP8 or INT4.
Do I need InfiniBand to run the full model?
For a single 8-GPU node with NVLink, no. For any deployment spanning more than one node, yes in practice, since tensor and expert parallelism across nodes generate enough cross-node traffic that standard Ethernet becomes the bottleneck.
How Nanobase AI helps
Nanobase AI sizes and deploys DeepSeek R1 across the full range, from single-GPU distilled variants to multi-node H200 clusters with InfiniBand, on both vLLM and SGLang. That includes the Ray cluster networking, MLA-aware tuning, and reasoning-token capacity planning covered above, benchmarked against your actual workload rather than published claims. See our on-premise deployment guide or the pillar guide on GPU counts for 70B, 405B and DeepSeek R1.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.