Skip to content
ToolHosted agent

KradleBench

Benchmark that scores frontier AI models in live multi-agent Minecraft worlds

KradleBench is an AI tool for research. It is available on Web.

Platforms
Web
Listing updated

Similar tools

About KradleBench

The README is absent, so this listing draws on the product's own site, which presents KradleBench as an evaluation suite rather than a tool for end users. Models are placed into shared, changing Minecraft worlds where other agents compete or cooperate, and their results are compared across collaboration, economy, conflict, logic and creativity.

The site contrasts this with static benchmarks: runs last several minutes, early choices affect later outcomes, and the world keeps moving while the model decides. Agents receive structured observations and act by writing code that moves their player through a 3D world. Each run can be replayed in 3D, and the site ranks models by score against LLM cost per run.

The benchmark is aimed at AI labs, researchers and teams comparing model behaviour in interactive settings. Access to the underlying data is by request, according to the site.

What it does

Multi-agent world scenarios
Models share live Minecraft worlds where they must cooperate or compete with other agents.
Five scored dimensions
Results are broken down into collaboration, economy, conflict, logic and creativity.
Interactive 3D replays
Each evaluation run can be played back and rotated in 3D to inspect model behaviour.
Cost versus performance chart
A Pareto frontier plots benchmark score against LLM cost per run for each model.
Human baseline comparison
Scores from a skilled human player are included alongside model results for comparison.
Code-driven agent actions
Agents receive structured observations and move through the 3D world by executing code.

Good for

  • Comparing frontier models on cooperation and competition in dynamic environments
  • Evaluating cost-effectiveness of LLMs against their benchmark scores
  • Studying how models adapt to changing rules and other agents over time
  • Reviewing individual model runs through replays to understand decision making

KradleBench FAQ

Is KradleBench open to the public?

The site says data access is available on request, and it offers a notification for newly added models.

What is measured in the benchmark?

Models are scored on collaboration, economy, conflict, logic and creativity inside live Minecraft worlds.

How long do benchmark runs take?

The site states that runs last 2 to 15 minutes.

What are the best alternatives to KradleBench?

The closest alternatives to KradleBench are ClawBench, Asiprize and Mirage.

Where this comes from

  • Description: written from the project's website.