# Attest

> Attest is an agent framework for developer tools. Testing framework for AI agents that checks replies, tool calls and state changes in CI. It is available on API, CLI and self-hosted.

- **Kind:** Agent
- **Platforms:** API, CLI, self-hosted
- **Website:** https://attest-framework.github.io
- **Categories:** [developer tools](https://ai-powered.com/categories/developer-tools)
- **Listing updated:** 2026-11-18

## About

Attest is a Python and TypeScript testing framework that runs with pytest or Vitest. Instead of only grading the text an agent returns, it inspects the sequence of tool calls and the changes an agent makes to external state. The README describes starting with deterministic checks and adding an LLM judge only where those checks cannot express a requirement.

The workflow is built around recording and replay. The capture step records a run's trace and model exchanges, applies redaction and drafts contracts from it. Strict replay then runs the agent's real code against those recorded model responses through a local proxy, so matching requests make no live model calls. The docs note that this removes model variability but not other sources of flaky tests, such as tool dependencies.

The project is aimed at teams that ship agents with tool access and want regression tests that fail a pull request when, for example, a refund tool is called twice for the same order. Checks report pass, fail or not_evaluated, and a CI policy can block merges on failures.

## Features

- **Deterministic assertion layers:** Checks on output schema, required tools and substrings run without an API key and at no cost.
- **Tool call contracts:** Behavior rules are written over the trace's tool calls, such as limiting how often a tool runs per order.
- **Strict replay:** Replays recorded model responses through a local proxy so the agent's code runs without live model calls.
- **CI merge gate:** A policy check decides whether failed or not-evaluated tests block a merge, and can be required in branch protection.
- **Capture from incidents:** Records a run's trace and model exchanges, applies redaction, and drafts contracts for review.

## Good for

- Adding regression tests for tool-call mistakes such as duplicate refunds
- Blocking agent changes in CI when recorded behavior contracts fail
- Testing agent code without spending on live model calls on every run
- Checking multi-step agent behavior with pytest or Vitest

## FAQ

### Which languages and test runners does it support?

The README says it supports Python with pytest and TypeScript with Vitest.

### Does it need an API key to run the example tests?

The README says the Python example's assertions are deterministic and need no API key, and the TypeScript test makes no provider calls.

### Which agent frameworks are covered?

The README lists adapters including OpenAI, Anthropic, Gemini, Ollama, LangChain, CrewAI, OpenTelemetry, Google ADK and LlamaIndex, with maturity tiers that vary by language.

### What are the best alternatives to Attest?

The closest alternatives to Attest are Kane CLI By TestMu AI, pydantic-ai, VSCode Extension, webqa-agent and harness-sdk.

## Where this comes from

- Description: written from the project's website.

## Alternatives

- [Kane CLI By TestMu AI](https://ai-powered.com/agents/kane-cli-by-testmu-ai): Kane CLI By TestMu AI is a freemium hosted agent for developer tools.
- [pydantic-ai](https://ai-powered.com/agents/pydantic-ai): pydantic-ai is an open-source agent framework for design and media.
- [VSCode Extension](https://ai-powered.com/agents/vscode-extension): VSCode Extension is an open-source agent for developer tools.
- [webqa-agent](https://ai-powered.com/agents/webqa-agent): webqa-agent is an open-source agent for browser automation.
- [harness-sdk](https://ai-powered.com/agents/harness-sdk): harness-sdk is an open-source agent framework for multi-agent frameworks.
- [AgentPilot](https://ai-powered.com/agents/agentpilot): AgentPilot is a freemium open-source agent for ai models and llm tooling.
- [Documentation](https://ai-powered.com/agents/documentation): Documentation is an open-source agent framework for developer tools.
- [Web](https://ai-powered.com/agents/web-6): Web is an open-source agent framework for multi-agent frameworks.

---
Source: https://ai-powered.com/agents/attest-4
