Gemma 3 27B is a dense model needing about 54 GB of VRAM in BF16, roughly 27 GB in FP8, and about 14 to 15 GB in INT4 for its weights, so the minimum practical GPU depends entirely on which precision is chosen. At FP8, a single 24 GB consumer GPU is too tight once KV cache is added, but a 32 GB or 48 GB card, or any data-center GPU like an L40S or H100, runs it comfortably with room for moderate concurrency and context length. At INT4, Gemma 3 27B fits on a single 24 GB GPU such as an RTX 4090 with meaningful headroom left for KV cache, making it a reasonable choice for single-workstation or small-team deployments. Gemma 3 also supports a long context window, so teams planning to use large portions of that context should budget extra KV cache beyond the baseline weight figure regardless of which quantization level is chosen. For production serving with multiple concurrent users, an RTX PRO 6000 or a single H100 gives more consistent headroom than a consumer card. Nanobase AI validates the exact minimum GPU configuration against a customer's expected context length and concurrency before recommending hardware.
Matching precision to what's actually on the market
The minimum GPU question only has a stable answer once precision is fixed, because Gemma 3 27B's weight footprint changes by nearly a factor of four between BF16 and INT4. The more useful framing is working backward from GPUs teams actually have access to or are considering buying, then checking which precision leaves workable KV cache headroom on each one, rather than starting from an abstract "minimum" that ignores concurrency entirely.
| Target GPU | VRAM | Precision that fits with headroom | Approx. KV cache room left |
|---|---|---|---|
| RTX 4090 / similar 24 GB consumer | 24 GB | INT4 (~15 GB weights) | ~8–9 GB, moderate concurrency |
| RTX PRO 6000 | 96 GB | FP8 or INT4 | 40–80 GB, high concurrency and long context |
| L40S | 48 GB | FP8 (~27 GB weights) | ~19 GB, moderate concurrency |
| H100 | 80 GB | BF16, FP8, or INT4 | 25–65 GB depending on precision chosen |
A 24 GB consumer GPU is only viable for Gemma 3 27B at INT4; at FP8 the 27 GB weight footprint alone exceeds a 24 GB card's total capacity, not just its headroom, which rules that combination out entirely rather than just making it tight.
Why context length changes this table
Gemma 3 supports a long context window, and KV cache usage grows linearly with both context length and concurrent sessions regardless of which precision the weights use. A single long-context session, tens of thousands of tokens, can consume several gigabytes of KV cache on its own, which matters most on the tighter end of this table, such as the 24 GB INT4 configuration, where a handful of long-context users can exhaust the remaining headroom even though the base weights fit comfortably. On the RTX PRO 6000 or H100 rows, the much larger remaining pool absorbs long-context usage with far less risk of hitting a ceiling.
Choosing between the three precision levels in practice
- INT4 for single-workstation or small-team deployments where the priority is running on the smallest, cheapest GPU available and a modest accuracy trade-off is acceptable for the use case.
- FP8 for production serving with meaningful concurrency on a 48 GB or larger GPU, striking a balance between weight footprint and the quality retention FP8 typically offers over more aggressive quantization, as discussed in does FP8 reduce model quality.
- BF16 only when GPU memory is not the binding constraint, such as on an H100 or larger card, and the deployment specifically wants to avoid any quantization-related quality question entirely.
The right minimum GPU for Gemma 3 27B is the smallest one that leaves enough KV cache headroom for the actual expected concurrency and context length, not the smallest one the weights alone fit onto.
How Gemma 3 27B compares to nearby model sizes on the same hardware
Gemma 3 27B's position just below the 32B tier is worth weighing against nearby dense models when the GPU budget is fixed and the model choice itself is still open.
| Model | BF16 weight size | FP8 weight size | INT4 weight size |
|---|---|---|---|
| Gemma 3 27B | ~54 GB | ~27 GB | ~14–15 GB |
| Generic 32B dense model | ~64 GB | ~32 GB | ~17–18 GB |
| Generic 14B dense model | ~28 GB | ~14 GB | ~8 GB |
On a fixed 24 GB or 48 GB budget, Gemma 3 27B's footprint sits close enough to the 32B tier that the same GPU choice generally covers both, while a 14B model leaves noticeably more headroom on the same card at any precision, a trade-off worth revisiting if the target GPU is smaller than what this table's Gemma 3 27B row assumes.
Frequently asked questions
Is INT4 quantization for Gemma 3 27B production-ready?
INT4 quantization of models in this size class is widely used in production today through formats like AWQ and GPTQ, generally with a small measurable quality difference from full precision on most tasks; testing against the specific target use case is worth doing before committing, as covered in AWQ vs GPTQ vs GGUF.
Does Gemma 3 27B need a data-center GPU, or is a consumer card enough?
A consumer card is enough for single-user or small-team use at INT4, but data-center GPUs like the L40S, H100 or a workstation card like the RTX PRO 6000 provide more reliable headroom for multi-user production serving with longer context windows.
How does Gemma 3 27B compare to a 32B or 70B model for the same GPU budget?
Gemma 3 27B sits in a sweet spot below the 32B and 70B tiers, needing meaningfully less memory at every precision level while remaining competitive on many tasks, making it a reasonable choice when a smaller GPU budget is a hard constraint.
Can two smaller GPUs run Gemma 3 27B instead of one larger one?
Yes, splitting a 27B model across two smaller GPUs with tensor parallelism is possible and sometimes more available or affordable than a single larger card, though it adds interconnect and configuration complexity that a single sufficiently large GPU avoids.
How Nanobase AI helps
Nanobase AI validates the exact minimum GPU configuration against a customer's expected context length and concurrency before recommending hardware, rather than quoting a generic minimum that ignores real usage patterns. This is part of Nanobase AI's broader work on on-premise LLM deployment across model sizes and precision levels.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.