Braintrust

Braintrust is an AI observability and evaluation platform for teams running LLM-based agents in production. It captures full traces of agent activity—including prompts, responses, tool calls, and retrieval steps—so teams can inspect what happened, score output quality, and catch regressions before they reach users.

Core modules include Observability (traces, spans, dashboards, alerts, and online scoring), Evals (experiments against versioned datasets with LLM, code, or human scoring), and Discovery (natural language trace investigation and pattern identification). Brainstore, the underlying database, provides log search across long prompts and responses, with Nitro as its query engine. An MCP server connects coding agents to the platform, and native SDKs are available for Python, TypeScript, Go, Ruby, C#, and others. The gateway offers a unified API across providers such as OpenAI, Anthropic, and Google. Additional features include Loop (an AI agent that generates prompts, scorers, and datasets), Patterns automation, custom trace views, and trace-to-dataset conversion.

Braintrust is packaged in three editions: a free Starter plan, a paid Pro plan, and a custom-priced Enterprise plan. Pricing scales with processed data volume, scores, and model credits, with unlimited users across all plans. Enterprise adds custom data retention, RBAC, premium support, and on-prem or hosted deployment options.