
Evaluate and iteratively improve AI agents on Google Cloud with an eval flywheel
- Open source
- 24.4K GitHub stars
- AI models and LLM tooling

Evaluate and iteratively improve AI agents on Google Cloud with an eval flywheel

Call Gemini, open models and custom endpoints on Google Cloud; debug 429 and 404 errors

Move an app from Gemini API in AI Studio to Google Cloud Agent Platform

Upload, list, update and delete models in the Agent Platform Model Registry

Create, list, version and delete managed prompts in Google Agent Platform via Python

List, inspect and query Agent Platform RAG Engine corpora and ground Gemini answers

Search, upload and manage skills in the Gemini Enterprise Agent Platform Skill Registry

Fine-tune open models or Gemini on Google Agent Platform, from dataset to endpoint

List, inspect and cancel GenAI tuning jobs in Google Agent Platform

Turn a REST API on Google Cloud API Gateway into a remote MCP server

Agent skill that distils a person's materials into a reusable profile for AI agents

Run a cloud-trial acceptance test on a GPU cluster and write a dated report

Practitioner runbook for training and serving large models, from GPUs to NaN loss spikes

Search the Hugging Face Hub and run GPU jobs through the HF MCP server tools

Drive the Hugging Face Hub from the hf CLI: repos, buckets, jobs, endpoints and papers

Plan a model deployment to Amazon SageMaker and pick the right inference pathway

Deploy SageMaker endpoints with autoscaling, alarms, tags and smoke tests built in

Choose the right SageMaker serving container and current image URI for a model

Estimate GPU memory for a Hugging Face model's weights and KV cache without downloading it

Recommend top Hugging Face models for a task from official leaderboards and device limits

Run local evals on Hugging Face models with inspect-ai or lighteval and pick a backend

Explore Hugging Face datasets through the read-only Dataset Viewer API

Build Gradio web UIs, chatbots and ML demos in Python, and call Spaces from the CLI

Fine-tune LLMs with TRL or Unsloth on Hugging Face Jobs GPUs and convert to GGUF

Find GGUF models on Hugging Face, pick a quant and serve them locally with llama.cpp

Open-source AI gateway in Go that routes LLM calls with budgets, fallbacks and MCP

Google's open-source Go toolkit for building and deploying AI agents

Python library that optimises prompts and code with reflective evolutionary search

Open-source infrastructure for asynchronous reinforcement learning on language models

Open-source Python framework for building and orchestrating multi-agent AI systems