Manifest

Manifest is an open source LLM gateway that helps builders and teams create reliable agents and workloads, managing inference consumption and coordinating routes between models and providers. It offers a self-hosted option and a cloud version, with support for local models and BYOK providers, enabling customization of routing rules and cost visualization. The gateway is designed to connect autonomous agents, apps, and coding assistants to various inference sources while tracking spend and performance.

Autofix repairs repairable 4xx errors on the fly before they reach the agent, and the gateway uses a healing service to patch requests prior to retry. The system prioritizes repair over fallback, and it can operate in cloud or self-hosted environments, with data handling designed to scrub secrets and protect credentials.

Manifest provides model fallbacks that automatically retry on another provider when one goes down, full body logs for every call, and observability features to inspect requests and responses. It supports self-hosted deployments, local models, bring-your-own-keys, and parameter configuration via a visual interface, along with cost visualization to monitor spending by route, agent, key, and provider.

The product is presented as open source, with options to deploy on cloud or on user infrastructure. It includes a Docker-based self-hosted version under the MIT license, supports local models like Ollama and llama.cpp, and offers plans ranging from a free cloud tier to enterprise deployments with SSO, audit logs, and a service level agreement.

Use cases

• AI builders connect subscriptions, BYOK providers and local models with their harness in seconds.