
Eval driven dev
SkillGuides an agent through building LLM eval pipelines with datasets, evaluators and pixie
- Open source
- 41.3K GitHub stars
- AI models and LLM tooling


Guides an agent through building LLM eval pipelines with datasets, evaluators and pixie

Lets your agent manage Arize annotation configs, review queues and span labels

Improves LLM prompts using trace data, eval scores and experiments in Arize

Guides building, running and validating LLM evaluators with the Phoenix framework

Guides agents through self-critique, reflection loops and LLM-as-judge evaluation

Adds Arize AX tracing to an LLM app in two phases: read-only analysis, then implementation

Runs a model over an Arize dataset, saves the results as an experiment and compares runs

Lets an agent create, list, update and delete Arize AI LLM provider integrations

Exports and inspects Arize traces, spans and sessions to debug what an LLM app is doing

Evaluate and iteratively improve AI agents on Google Cloud with an eval flywheel

Build custom evaluation metrics that grade your AI agent the way you do

Creates, exports and appends to Arize evaluation datasets using the ax command line
The closest alternatives to Arize evaluator are Eval driven dev, Arize annotation, Arize prompt optimization, Phoenix evals and Agentic eval.
Yes. Eval driven dev, Arize annotation, Arize prompt optimization have a free or open-source option.