Opik

What is Opik?

Opik is an open source LLM observability and evaluation platform built by Comet. You instrument your AI app or agent to capture every LLM call, tool execution, and retrieval step as a nested, inspectable trace, then layer on automated evaluation, guardrails, prompt management, and a playground — so you can see what your model did, score how well it did it, and catch problems before and after you ship.

What is Opik best for?

Teams building and running LLM apps and multi-step agents who need production visibility and rigorous evaluation on their own infrastructure. It fits engineers who want framework-agnostic tracing, LLM-as-a-judge scoring, and CI-gated regression testing without routing sensitive traces through a hosted-only service — especially those already using LangChain, LlamaIndex, CrewAI, or LangGraph.

What can Opik do?

• Trace LLM calls, tool executions, retrieval, and agent steps as nested spans, with feedback scores added via SDK or UI
• Run evaluations with LLM-as-a-judge metrics — hallucination, moderation, and RAG relevance — plus code-based and heuristic checks
• Manage datasets and experiments to benchmark changes before shipping
• Gate regressions in CI with a PyTest integration
• Apply Opik Guardrails to enforce safe inputs and outputs in production
• Improve prompts and agents automatically with the Agent Optimizer SDK
• Iterate on prompts and model configs interactively in a Prompt Playground
• Integrate with 70+ frameworks and providers (OpenAI, Anthropic, LangChain, LlamaIndex, CrewAI, Dify) via Python/TypeScript SDKs and OpenTelemetry

Is Opik free?

Yes — Opik is fully free to self-host under the Apache 2.0 license, with unlimited spans and retention; you only pay for your own infrastructure. Comet also offers a managed cloud: a Free tier ($0/mo, 25k spans/month, 60-day retention, up to 10 members), Pro at $19/mo (100k spans, up to 50 members, customizable limits), and custom Enterprise pricing. You pay for managed hosting and enterprise compliance, not the core software.

Where does Opik fall short?

• The whole platform is Apache 2.0 — unusually permissive for this category — but that also means Comet monetizes through managed cloud and enterprise support rather than gated features, so on-prem teams are largely on their own for scaling and SLAs.
• A production self-host is a service-oriented stack, not a single container: it expects ClickHouse (traces), MySQL (metadata), Redis (caching and queues), and S3-compatible object storage, typically run on Kubernetes via Helm.
• Evaluation and observability are its focus, not full-stack infrastructure monitoring. If you need application and infra APM alongside LLM traces, Datadog covers far more ground.

What does Opik replace?

Opik is a self-hosted, open source alternative to LangSmith, LangChain’s hosted observability and evaluation platform; to Braintrust for LLM evaluation and monitoring; and to Weights & Biases Weave for LLM tracing. It also covers the LLM-observability slice of Datadog. Among open source options it sits alongside Langfuse and Arize Phoenix.

FAQ

Is Opik open source? Yes — the entire platform is licensed Apache 2.0, including the backend and web UI, not just the client SDKs. That’s more permissive than the open-core model many competitors use, and it lets you self-host every feature at no cost.

Can I self-host Opik for free? Yes. Self-hosting is free with unlimited spans and retention under Apache 2.0; you only pay for the servers. Docker Compose gets you running locally in minutes; production uses a Kubernetes Helm chart.

Is Opik a good LangSmith alternative? For teams that want to self-host and avoid per-seat pricing, yes. Unlike LangSmith — which reserves self-hosting for Enterprise — Opik is framework-agnostic and free to run in full, with comparable tracing, evaluation, and prompt tooling.

What do I need to run Opik? For local testing, Docker Compose. For production, plan for ClickHouse, MySQL, Redis, and S3-compatible object storage, usually deployed on Kubernetes with the official Helm chart.