Bifrost

What is Bifrost?

Bifrost is an open source LLM gateway that sits between your application and the model providers, exposing a single OpenAI-compatible API for all of them. Built in Go by Maxim AI, it routes each request to the right provider, retries and fails over when one is down, and adds only microseconds of overhead per call.

What is Bifrost best for?

Teams running LLM calls across several providers who want one place to manage keys, budgets, and routing without rewriting client code. It suits high-throughput production traffic — the gateway adds roughly 11 microseconds at 5,000 requests per second — and works as a drop-in base-URL swap for the OpenAI, Anthropic, LangChain, or Vercel AI SDKs.

What can Bifrost do?

• Expose one OpenAI-compatible API across OpenAI, Anthropic, AWS Bedrock, Google Vertex, Azure, Mistral, Cohere, Groq, and local models served by Ollama or vLLM — 1,000+ models in total
• Fail over and load-balance automatically across providers and API keys when a model errors or rate-limits
• Cut cost and latency with semantic caching that reuses responses for similar requests
• Govern spend with virtual keys, teams, and hierarchical budgets, plus rate limiting and usage tracking
• Connect external tools through a built-in Model Context Protocol (MCP) gateway
• Handle text, image, and audio requests, including streaming, through the same interface
• Ship with a web UI for configuration and live monitoring, plus OpenTelemetry and Prometheus metrics
• Extend behavior with custom plugins, and drop it in via npx, Docker, or the Go SDK

Is Bifrost free?

Yes — the Bifrost gateway is open source under the Apache-2.0 license and free to self-host, with fallbacks, semantic caching, virtual keys and budgets, MCP, observability, and the web UI all included. A paid Enterprise plan (custom pricing, 14-day trial) adds SSO and RBAC, cluster mode and adaptive load balancing, guardrails, vault integrations, and audit logs. You bring your own provider keys, so there is no per-token markup.

What does Bifrost replace?

Bifrost is a self-hosted alternative to hosted LLM routers like OpenRouter and commercial gateway platforms like Portkey. It does the same unify-every-provider-behind-one-API job, but runs on your own infrastructure with your own provider keys — so you aren’t paying a per-token markup or routing production traffic through someone else’s service. Developers migrating off LiteLLM also pick it up for the lower request overhead.

Where does Bifrost fall short?

• It supports fewer providers than LiteLLM. Bifrost covers around a dozen major providers (and 1,000+ models), while LiteLLM integrates 100+, so a niche or long-tail provider may not be built in yet.
• It’s open-core. The gateway itself is fully open source, but the features larger orgs often need — SSO and RBAC, cluster mode, guardrails, vault-backed secrets, and audit logs — are gated behind the paid Enterprise plan.
• For a single-provider setup it’s overkill. The payoff is routing, fallback, and governance across many providers and keys; with one OpenAI account and no budget or governance needs, a gateway just adds a hop.

FAQ

Is Bifrost open source? Yes. The Bifrost gateway is released under the Apache-2.0 license, a permissive OSI-approved license, with the full source on GitHub. Some enterprise features (SSO, RBAC, clustering, guardrails, audit logs) are proprietary add-ons, but the core gateway is genuinely open source.

Can I self-host Bifrost for free? Yes. Self-hosting is free — start it with npx -y @maximhq/bifrost, a single Docker container, or the Go SDK. You only pay for your own server and the provider API usage you route through it.

Is Bifrost a good OpenRouter alternative? For teams that want to keep their own provider keys and avoid a per-token markup, yes — you get the same one-API-for-every-model experience, self-hosted. OpenRouter is simpler if you’d rather not run infrastructure and want instant access to models without opening your own provider accounts.

What do I need to run Bifrost? A server or container host, plus API keys for the providers you want to route. Bifrost ships as a single Go binary you can launch with npx or Docker; for production add your usual container orchestration, and an OpenTelemetry or Prometheus stack if you want to export its metrics.