For on-premise Kubernetes deployment, Qdrant and Milvus are generally the strongest vector database choices, since both ship official Helm charts and operators designed for production Kubernetes environments with horizontal scaling, persistent storage, and rolling upgrades. Milvus separates compute and storage into distinct components, which fits naturally into a Kubernetes architecture and scales well for very large collections spread across multiple nodes, though it brings more operational components, such as etcd and object storage dependencies, that need to be managed. Qdrant runs as a simpler, single-binary service that is easier to operate and monitor within Kubernetes for small to mid-sized teams, with less infrastructure overhead but somewhat less mature distributed sharding than Milvus at very large scale. Weaviate is also viable on Kubernetes and adds built-in hybrid search, though its resource footprint per node tends to run higher. For teams already running Postgres in Kubernetes, pgvector avoids introducing a new stateful service entirely, at the cost of the scale and feature ceiling dedicated vector databases offer. The right choice depends on the team's existing Kubernetes operational maturity and the target scale more than on any generic benchmark. Nanobase AI deploys and operates vector databases on customer Kubernetes clusters, including GPU Operator-managed nodes for embedding and reranking workloads.
Operational maturity matters more than benchmark numbers
Feature comparisons between vector databases tend to converge on similar answers as of 2026, since the major options now all support approximate nearest neighbor search, metadata filtering, and some form of hybrid search. What actually differentiates them for a Kubernetes deployment is operational maturity: how the database behaves during a node failure, how backups and restores work under a Kubernetes-native workflow, and how much dedicated platform engineering time it demands to run reliably at 2 a.m. when something goes wrong. Choosing a vector database for Kubernetes should weigh operational burden as heavily as raw query performance, since the team running it on-call cares more about the former.
Operational comparison table
| Dimension | Qdrant | Milvus | Weaviate | pgvector |
|---|---|---|---|---|
| Deployment model | Single binary, official Helm chart | Distributed, separates compute and storage | Single service, official Helm chart | Extension on existing Postgres |
| High availability | Built-in clustering | Strong at large scale, more moving parts | Built-in clustering | Inherits Postgres HA setup |
| Backup and restore | Snapshot-based, straightforward | Requires backing up object storage and metadata store together | Snapshot-based | Standard Postgres backup tooling |
| Additional infra dependencies | Minimal | etcd, object storage (e.g. MinIO or S3-compatible) | Minimal | None beyond Postgres itself |
| GPU acceleration fit | Runs alongside GPU nodes for embedding workloads | Same, plus GPU-accelerated index building at scale | Runs alongside GPU nodes | Same, no native GPU acceleration |
| Best operational fit | Small to mid-sized teams wanting low overhead | Teams needing very large scale and already running distributed infra | Teams wanting built-in hybrid search out of the box | Teams already running Postgres who want to avoid a new stateful service |
Key takeaway: Qdrant and Weaviate minimize new operational surface area, Milvus trades more moving parts for scale headroom, and pgvector avoids a new database entirely for teams already running Postgres.
Deploying with GPU Operator-managed node pools
A common pattern runs the vector database on general-purpose nodes while embedding and reranking inference runs on GPU nodes managed by the NVIDIA GPU Operator, connected over the cluster's internal network so the vector database never needs GPU access itself. Node affinity and taints keep the vector database's pods off GPU-equipped nodes, reserving that capacity for actual inference workloads, while the vector database's own resource requests are tuned for memory, since index size in memory is usually the binding constraint for query latency rather than CPU.
A minimal Helm-based deployment for Qdrant, as a representative example, looks like:
helm repo add qdrant https://qdrant.github.io/qdrant-helm
helm install qdrant qdrant/qdrant \
--set persistence.size=200Gi \
--set replicaCount=3 \
--namespace rag-platformKey takeaway: keep the vector database on general-purpose nodes and reserve GPU-equipped nodes exclusively for embedding, reranking, and generation inference.
What breaks operationally at scale
Teams that move a vector database from a proof-of-concept deployment to production on Kubernetes most often get surprised by three things: memory pressure as the index grows beyond what was originally provisioned, since most of these databases perform best with the index resident in memory rather than read from disk; backup and restore procedures that were never actually tested end to end until a real incident forced it; and rolling upgrades that require careful sequencing to avoid a brief period of degraded query results while replicas resync. Testing a full backup-and-restore cycle and a rolling upgrade in a staging environment before go-live catches most of these issues before they become an incident.
Key takeaway: test backup-and-restore and rolling-upgrade procedures in staging before go-live, since these are the operational failure modes that surface first in production, not query latency.
Frequently asked questions
Does Milvus's extra complexity pay off at smaller scale?
Usually not. Milvus's separation of compute and storage pays off most clearly at very large collection sizes spread across many nodes; for a mid-sized corpus, the additional operational components like etcd and object storage often add more overhead than the scale benefit justifies.
Can pgvector handle production RAG workloads?
Yes, for many enterprise-scale corpora, particularly when a team already operates Postgres reliably and wants to avoid a new stateful service. Its scale and specialized feature ceiling, such as advanced quantization, is lower than purpose-built vector databases, so very large or highly latency-sensitive workloads may outgrow it.
How do we choose between Qdrant and Weaviate specifically?
Both are close in operational profile; Weaviate ships more built-in hybrid search and schema features out of the box, while Qdrant tends to run with a lighter resource footprint per node. For teams without a strong preference, a short proof-of-concept against real data is more informative than a feature comparison alone.
Does the vector database need its own GPU?
No, standard approximate nearest neighbor search runs efficiently on CPU for most enterprise-scale corpora. GPU acceleration for vector search exists for specialized very-high-throughput cases, but the more common GPU need in a RAG stack is for the embedding, reranking, and language model inference steps.
How Nanobase AI helps
Nanobase AI, a Silicon Valley enterprise AI engineering company, deploys and operates vector databases on customer Kubernetes clusters alongside GPU Operator-managed inference nodes, including the backup, upgrade, and monitoring procedures that keep them reliable past the initial rollout. Compare this against Elasticsearch or OpenSearch as a vector store if you already run one of those platforms, or see our solutions for the full stack.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.