OpenLIT

OpenLIT is an open-source agent harness engineering platform that provides OpenTelemetry-native observability for AI agents. It supports evals, guardrails, prompt management, and cost and GPU monitoring, and can be self-hosted under the Apache 2.0 license. The tool is designed for developers and engineers who build and operate AI agents and coding agents, offering traces, evaluation pipelines, and dashboards in a single platform.

OpenLIT traces, evaluates, guards, and improves everything around the model in production. It emphasizes production data usage, not proprietary telemetry formats, and supports tracing, prompt versioning, and cost visibility across LLMs and agent workflows.

The platform combines agent observability, evaluation, guardrails, and prompt management with Vault, OpenGround, and Otter components. It is self-hosted and aims to keep data ownership, while providing OpenTelemetry-native instrumentation that can export to Grafana, Datadog, or any OTLP backend.

OpenLIT is built to scale with features like a ClickHouse-backed storage layer, OpenTelemetry-native traces and metrics, and GPU monitoring during LLM inference. It supports Docker Compose and Kubernetes Helm deployment, and invites community contributions via GitHub.

Use cases

• Trace, evaluate, guard, and improve AI agents with production data without locking telemetry into a proprietary format
• Self-hosted observability for cost and GPU visibility across LLMs and agent workflows