Opik

Opik is an AI observability and evaluation platform built by Comet for agentic systems. It helps developers test, ship, and continuously improve LLM applications and autonomous agents, covering use across individual developers and large organizations. The platform is compared by its maker against alternatives such as LangSmith, Langfuse, Arize, Braintrust and Galileo.

Opik logs each step an agent takes, from user interactions to context retrieval and tool calls, and lets a team define what a good outcome looks like with a reference dataset or a plain-text assertion. Traces from development, testing or production can be scored and compared across agent versions, and the core feature set is available as open-source code that can be downloaded from GitHub and run locally.

The platform scores performance with over 30 metrics covering answer relevance, context precision, task completion and hallucination, and it can alert in real time when a production interaction fails a test criterion. Guardrails block content and policy violations and protect against PII exposure. A Prompt Optimizer offers several optimization algorithms, and a coding assistant called Ollie suggests and implements fixes directly in an agent's codebase.

Opik is available as a free open-source download, a Free Cloud plan with up to 10 team members and 25k spans a month, a Pro Cloud plan at $19 per month with up to 50 team members and 100k spans a month, and a custom Enterprise plan with unlimited team members and compliance certifications including SOC 2 and HIPAA. A REST API and Python client work with both the open-source and cloud versions.

Use cases

• Surfacing silent errors with recommended fixes during debugging
• Comparing agent versions across development, testing and production before shipping
• Controlling coding agent spend across engineering teams
• Producing audit logs for a governance team