Vision-language models like Qwen 3 VL generally need roughly the same base VRAM as a text-only model of the same parameter count, plus additional memory for the vision encoder and, more significantly, for the extra tokens that each image contributes to the context window once it is encoded and passed into the language model. A single high-resolution image can add anywhere from several hundred to a few thousand tokens depending on the model's vision tokenizer, which means KV cache usage grows noticeably faster in multimodal workloads than in equivalent text-only conversations, especially when a conversation includes multiple images or documents. For a mid-size vision-language model in the 7B to 32B range, this typically means budgeting meaningfully more KV cache headroom than the text-only sizing formula would suggest, particularly for document understanding or multi-image use cases. Quantizing the language model portion to FP8 or INT4 still applies and helps offset the added image-token cost, though the vision encoder itself is sometimes kept at higher precision. Testing with actual image resolution and count per request gives a far more accurate memory estimate than a text-only formula. Nanobase AI, a Silicon Valley-based enterprise AI engineering company, accounts for image-token overhead explicitly when sizing GPUs for vision-language deployments.
The part of VLM sizing that a text-only formula misses
A vision-language model's weights follow the same rules as a text-only model of equal size: roughly 2 bytes per parameter in BF16, 1 in FP8, about 0.5 to 0.6 in INT4. What a text-only sizing formula misses entirely is that every image in a conversation gets converted into a block of tokens by the vision encoder before it ever reaches the language model, and those tokens sit in KV cache exactly like text tokens do for the rest of the conversation. The vision encoder itself is a small, fixed memory cost; the recurring cost that actually drives GPU sizing is the image-token tax on every request that includes an image.
Working through the image-token math
A single image can resolve to anywhere from roughly 250 tokens for a low-resolution thumbnail-style encoding to well over 2,000 tokens for a high-resolution image processed at full detail, depending on the model's vision tokenizer and the resolution setting used at inference time. Multiply that by the same per-token KV cache cost used for text (bytes per token depends on layer count, KV heads and head dimension) and a single high-resolution image can consume as much KV cache as several paragraphs of text.
| Scenario | Approx. image tokens | Relative KV cache impact vs. text-only turn |
|---|---|---|
| Low-res single image | ~250–500 | Similar to a short paragraph |
| Standard single image | ~500–1,200 | Similar to a long paragraph |
| High-res single image | ~1,500–2,500+ | Similar to a full page of text |
| Multi-image or multi-page document | 4,000+ | Dominates the conversation's context budget |
A document-understanding workload processing multiple scanned pages per request behaves, from a memory-sizing standpoint, much closer to a long-context text workload than a typical chat turn.
Sizing a mid-size VLM deployment
For a dense vision-language model in the 7B to 32B range, base weight sizing follows the same table used for any text model of that size. The adjustment is on top: budget KV cache assuming a higher effective token count per turn than the visible text would suggest, especially for screenshots, scanned documents, or multiple images per conversation.
- Estimate weight footprint using the target precision (BF16, FP8 or INT4) exactly as for a text-only model of the same parameter count.
- Estimate typical images per request and resolution setting, since this is usually configurable and directly controls token count.
- Convert expected image count to an effective token addition using the ranges above, then add that to the text-only KV cache estimate.
- Quantize the language-model side to FP8 or INT4 to offset the added image-token cost, since the vision encoder is sometimes kept at higher precision but is small enough that this rarely matters much.
- Load-test with actual production-resolution images, not placeholder text, since the token math above is model-specific and varies by vision tokenizer.
A text-only sizing formula for a VLM will reliably underestimate KV cache demand, sometimes significantly, once real images enter the workload, which is the single most common surprise teams hit when moving from a chat pilot to a document-heavy production use case.
Matching GPU choice to expected image volume
The GPU decision for a VLM deployment should be made against expected image volume per session, not base parameter count alone, since a light single-image chat use case and a heavy multi-page document pipeline need very different headroom on the same underlying model.
| Workload profile | Base model size | Recommended precision | Realistic GPU choice |
|---|---|---|---|
| Occasional single-image chat | 7B–14B | FP8 | Single L40S or H100 with ample headroom |
| Frequent single-image chat, moderate concurrency | 7B–14B | FP8 or INT4 | Single H100 or RTX PRO 6000 |
| Document understanding, multi-page requests | 14B–32B | FP8 | H100 or H200, sized for long effective context |
| High-volume multimodal agent pipelines | 32B+ | FP8, INT4 for the language backbone | Multi-GPU node with generous KV cache headroom |
Moving down this table trades a larger memory budget for handling heavier image volume without hitting the KV cache ceiling described above; the jump from single-image chat to document pipelines is usually where a mid-range GPU stops being enough.
Frequently asked questions
Do vision-language models need a different GPU than text-only models of the same size?
Not a different GPU type, but meaningfully more KV cache headroom for the same parameter count and expected concurrency, since image tokens inflate effective context length. The same H100, H200 or RTX PRO 6000 options used for text-only sizing still apply, just with a larger memory budget.
Is the vision encoder itself expensive in GPU memory?
No, the vision encoder is typically a small fraction of total model parameters and adds a modest, mostly fixed memory cost. The recurring cost comes from the tokens it produces per image, not from the encoder's own weight size.
Can lowering image resolution reduce GPU memory needs?
Yes, most VLM serving setups allow configuring maximum image resolution or tile count, and reducing it directly reduces token count and therefore KV cache usage, at some cost to the model's ability to read fine detail in the image.
Does multi-image or multi-page document processing change the GPU count needed?
It can, since multi-image requests behave like long-context requests. For high-volume document processing, budgeting the same way as a long-context text deployment, with more GPUs or higher-memory cards, is more accurate than sizing against typical single-image chat use.
How Nanobase AI helps
Nanobase AI, a Silicon Valley-based enterprise AI engineering company, accounts for image-token overhead explicitly when sizing GPUs for vision-language deployments, testing against actual production image resolution and volume rather than text-only assumptions. This includes document-understanding and multimodal agent workloads sized alongside broader enterprise integration needs.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.