vLLM serves LoRA adapters by loading one base model into GPU memory and dynamically applying different low-rank adapter weights per request, so dozens of fine-tuned variants can share a single deployment instead of each needing its own full model copy. You start the server with enable-lora set and register adapters using the lora-modules flag, pointing each adapter name to its checkpoint path, or add them dynamically at runtime through the API without restarting the server; each incoming request then specifies which adapter to use in the model field of the chat completions call. This is far more memory-efficient than full fine-tuning per use case, since LoRA adapters are typically tens to a few hundred megabytes compared with tens of gigabytes for a full model copy, and vLLM can hold many adapters resident while swapping which one is active per batch with minimal overhead. There are limits worth planning around, including maximum adapter rank, the maximum number of adapters loaded simultaneously, and a small per-request latency cost for adapter switching within a batch. This pattern fits well for multi-tenant products where each customer or use case has its own lightly fine-tuned variant of the same base model. Nanobase AI implements multi-LoRA serving for customers running many fine-tuned variants from shared GPU capacity.
The deployment shape, not just the concept
The value of serving LoRA adapters through vLLM is well understood: one base model in memory, many lightweight fine-tuned variants swapped in per request. What matters for an actual deployment is the specific sequence of steps and the limits that determine how many adapters and how much traffic that pattern can realistically support.
Plan the deployment around three numbers before writing any configuration: expected adapter count, adapter rank, and peak concurrent adapter diversity per batch, since each of these affects memory and latency differently.
Step-by-step deployment
- Start the vLLM server with LoRA support enabled and a maximum adapter count set to comfortably exceed your expected number of active adapters.
- Register known adapters at startup by pointing each adapter name to its checkpoint path, so they are available immediately without a runtime registration step.
- For adapters added after launch, use the dynamic adapter loading API to register new ones without restarting the server, which matters for multi-tenant products onboarding new customers continuously.
- Route each incoming request by specifying the target adapter name in the
modelfield of the chat completions call, exactly as you would specify a different model name. - Monitor per-adapter request latency separately, since adapter-switching overhead within a batch, while small, is not zero and can matter at very high adapter diversity per batch.
- Set eviction or a maximum resident adapter limit if your total adapter count could exceed available memory, so infrequently used adapters can be swapped out rather than causing an out-of-memory failure.
vllm serve meta-llama/Llama-3.1-8B-Instruct \
--enable-lora \
--lora-modules customer-a=/adapters/customer-a customer-b=/adapters/customer-b \
--max-lora-rank 32 --max-loras 8The memory math that determines how many adapters actually fit
A LoRA adapter is typically tens to a few hundred megabytes, depending on rank and which layers it targets, compared with tens of gigabytes for a full model copy at the same parameter count. This gap is what makes the pattern attractive: doubling your adapter count costs a small, predictable amount of additional memory, while doubling the number of fully fine-tuned model copies would double your entire GPU footprint. The limits worth tracking explicitly are the maximum adapter rank the server is configured for, the maximum number of adapters resident in memory simultaneously, and how many distinct adapters can be active within a single batch without a meaningful latency penalty from switching overhead.
Where this pattern fits, and where it doesn't
| Scenario | Multi-LoRA fit |
|---|---|
| Many customers, each with a lightly fine-tuned variant of the same base model | Strong fit, this is the primary use case |
| A handful of customers needing substantially different model behavior or capability | Weaker fit; consider whether full fine-tuning or different base models is more appropriate |
| Frequent onboarding of new customer-specific adapters | Strong fit with dynamic loading, avoids restart-per-onboarding |
| Adapters requiring very high rank for complex behavioral changes | Check memory math carefully; high-rank adapters approach the point where the efficiency advantage narrows |
Validating before production traffic
Test the deployment specifically for the scenario where a single batch contains requests for several different adapters simultaneously, since this is the condition that exercises adapter-switching overhead most heavily and is easy to miss if testing only ever sends requests for one adapter at a time. Confirm output quality per adapter against your fine-tuning validation set as well, since the deployment pattern doesn't change model behavior, but any regression should be caught before it reaches customers who each depend on a different adapter's specific behavior. This pattern pairs naturally with serving multiple models on one GPU server when some workloads need full model separation and others fit the LoRA pattern.
Frequently asked questions
Does vLLM support hot-swapping LoRA adapters without downtime?
Yes, adapters can be registered dynamically through the API after the server is running, which allows onboarding a new customer's adapter without restarting the server or interrupting traffic for existing adapters already in use.
Is there a performance cost to using LoRA adapters instead of a fully fine-tuned model?
The base model computation is identical; the adapter itself adds a small additional computation per forward pass, and switching adapters within a batch adds a modest overhead, both of which are typically much smaller than the memory and deployment savings gained from not duplicating the full model per variant.
What happens if we exceed the configured maximum number of LoRA adapters?
The server will reject requests for adapters beyond the configured maximum unless an eviction policy is in place; plan the maximum adapter count with real headroom above expected usage, and monitor for rejected requests as a signal that the limit needs raising.
Can different LoRA adapters have different ranks in the same deployment?
The server is generally configured with a maximum rank that individual adapters must not exceed, but adapters below that maximum can have varying ranks; check your specific vLLM version's documentation for exact behavior around mixed-rank adapter sets.
How Nanobase AI helps
Nanobase AI implements multi-LoRA serving deployments for customers running many fine-tuned variants from shared GPU capacity, including adapter memory planning, dynamic onboarding pipelines, and load testing under realistic multi-adapter batch diversity.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.