LLM Inference & Serving

Open source LLM inference and serving engines that run large language models on your own GPUs or CPUs — with high-throughput batching, OpenAI-compatible APIs, and quantization, so you can self-host open models instead of paying per token for a hosted API.