Milvus

What is Milvus?

Milvus is an open source vector database built for GenAI and similarity search. It stores embeddings from text, images, audio, and other unstructured data, then runs fast approximate-nearest-neighbor search over them — the retrieval layer behind RAG, semantic search, and recommendations. It’s designed to scale to tens of billions of vectors.

What is Milvus best for?

Teams running vector search at serious scale — hundreds of millions to billions of embeddings — who need horizontal scaling, GPU acceleration, and fine-grained index control without paying per query for a managed API. It fits production RAG, multimodal, and recommendation systems where a single-node database would run out of room.

What can Milvus do?

• Scale to billions of vectors with a distributed, Kubernetes-native architecture that separates compute from storage
• Choose from multiple index types — HNSW, IVF, FLAT, SCANN, DiskANN — with quantization and memory-mapped variants
• Run hybrid search combining dense vectors, sparse vectors, and native full-text (BM25) search with metadata filtering
• Accelerate search on CPU or GPU for high-throughput workloads
• Deploy three ways: Milvus Lite (a pip-installable library), Standalone (single machine), or Distributed (clustered)
• Integrate with LangChain, LlamaIndex, OpenAI, Hugging Face, DSPy, and Haystack
• Secure access with role-based access control, TLS, and multi-tenancy via databases, collections, and partitions

Where does Milvus fall short?

• The distributed mode is operationally heavy. A full deployment runs a cluster of microservices — coordinators, query/data/index nodes, a proxy, plus etcd and object storage (MinIO/S3) — so you’re maintaining many moving parts, not one binary. Simpler tools like Qdrant or Chroma ship as a single service.
• It’s overkill below scale. If you’re storing a few million vectors, Milvus’s architecture is more than you need; a lighter vector database is faster to run and reason about until you actually hit hundreds of millions of vectors.
• Filtered search has sharp edges. Milvus defaults to post-filtering, which can degrade recall on highly selective metadata filters unless you tune the query, and its Go/C++ core can see latency spikes under heavy concurrent write load.

Is Milvus free?

Yes — Milvus is fully open source under the Apache-2.0 license and free to self-host in any of its three modes; you only pay for the infrastructure it runs on. Zilliz Cloud, from the company behind Milvus, is the paid managed service (serverless, dedicated, and bring-your-own-cloud tiers) for teams that don’t want to run the cluster themselves.

What does Milvus replace?

Milvus is a self-hosted alternative to managed vector and AI-search services. It stands in for Pinecone and Azure AI Search as the vector store behind RAG and semantic search, and for teams outgrowing keyword search it’s an open source Algolia alternative that adds vector and hybrid retrieval — all on your own hardware, without usage-based, per-query billing.

FAQ

Is Milvus open source? Yes, fully. Milvus is released under the permissive Apache-2.0 license, so you can self-host, modify, and use it commercially without the source-available restrictions some tools carry.

Can I self-host Milvus for free? Yes. All three deployment modes — Lite, Standalone, and Distributed — are free; you only pay for your own servers or Kubernetes cluster. Zilliz Cloud is the paid managed option.

Is Milvus a good Pinecone alternative? For large-scale or self-hosted workloads, yes — it matches Pinecone’s core vector search with more index options and no per-query pricing. If you want zero operations, a managed service like Pinecone or Zilliz Cloud is simpler to start with.

What do I need to run Milvus? For experimentation, pip install pymilvus[milvus-lite] runs Milvus Lite in a Python process. Standalone needs Docker; the Distributed mode expects Kubernetes plus etcd and S3-compatible object storage.