# Kermt monitor

> Kermt monitor is an open-source agent skill for ai models and llm tooling. Checks progress of a detached KERMT training run: container state, logs and metrics. It is available on Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Windsurf and Goose.

- **Kind:** Skill
- **Pricing:** Open source
- **Platforms:** Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Windsurf, Goose
- **Website:** https://github.com/NVIDIA/skills/tree/main/skills/bionemo-kermt-monitor
- **Repository:** https://github.com/nvidia/skills
- **GitHub stars:** 4,037
- **License:** Apache-2.0
- **Skill name:** kermt-monitor
- **Collection:** https://github.com/NVIDIA/skills
- **Categories:** [AI models and LLM tooling](https://ai-powered.com/categories/ai-and-llm)
- **Listing updated:** 2026-11-18

## About

This skill gives the agent a way to inspect a KERMT job that was launched in the background, such as a pretraining or finetuning run. It reads the run directory's run.json manifest, asks docker for the container's status, and locates the relevant log file based on the workflow recorded in the manifest.

From the log it pulls out the latest progress lines, such as epoch, step and validation loss for pretraining, or fold, epoch and validation metric for finetuning, and estimates remaining time from the recorded start time. When a finetuning run has finished cleanly and a test results file exists, it also assembles a per-task test metrics table. The skill can print recent log lines, follow the live log, or emit a structured JSON status report. It is designed to only read data and never stops or modifies the container unless the user asks.

It expects docker and jq to be installed and a run directory that contains run.json, or a container name supplied directly. The SKILL.md notes that the manifest does not yet record the container name, so the skill falls back to matching run directories by modification time or to an explicit container argument.

## When your agent uses it

> Check progress for a detached KERMT run (pretrain, finetune, or any kermt_run_detached invocation). Reads run.json, queries docker for container state, tails the pretrain/finetune log, and parses progress lines (epoch, step, val loss).

## What it can access

From a static check of the files on GitHub; nothing was run.

- [Instructions only, no scripts](https://github.com/NVIDIA/skills/tree/HEAD/skills/bionemo-kermt-monitor)
- [License: Apache-2.0](https://github.com/NVIDIA/skills/tree/HEAD/skills/bionemo-kermt-monitor)

## Features

- **Container state check:** Queries docker for whether the run's container is running, exited or removed.
- **Training progress summary:** Parses recent log lines into epoch, step, fold and validation metric summaries.
- **Live log following:** Streams container logs until interrupted, useful for watching loss during a run.
- **Final test metrics table:** Builds a per-task metrics table from finetuning test results after a clean exit.
- **Structured JSON output:** Emits a single JSON status report so a parent agent can act on the result.
- **Read-only inspection:** Reads the manifest and logs without modifying run data or stopping containers.

## Good for

- Check how far a detached KERMT pretraining run has progressed
- Show the validation loss trend of a running finetune
- Follow the live training log until the run finishes
- Summarize per-task test metrics after a finetuning run completes
- Get the stop, remove and replay commands for a background run

## Install

### Claude Code

Copy the skill folder to `~/.claude/skills/kermt-monitor` to use it in all your projects:

```sh
tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/NVIDIA/skills.git "$tmp" &&
mkdir -p ~/.claude/skills &&
rm -rf ~/.claude/skills/kermt-monitor &&
cp -R "$tmp/skills/bionemo-kermt-monitor" ~/.claude/skills/kermt-monitor &&
rm -rf "$tmp"
```

For one project only, put the folder in `.claude/skills/kermt-monitor` inside it instead.

### Codex

Copy the skill folder to `~/.agents/skills/kermt-monitor` to use it in every repository:

```sh
tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/NVIDIA/skills.git "$tmp" &&
mkdir -p ~/.agents/skills &&
rm -rf ~/.agents/skills/kermt-monitor &&
cp -R "$tmp/skills/bionemo-kermt-monitor" ~/.agents/skills/kermt-monitor &&
rm -rf "$tmp"
```

For one repository only, put the folder in `.agents/skills/kermt-monitor` inside it instead.

### Cursor

Copy the skill folder to `~/.cursor/skills/kermt-monitor` to use it in every project:

```sh
tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/NVIDIA/skills.git "$tmp" &&
mkdir -p ~/.cursor/skills &&
rm -rf ~/.cursor/skills/kermt-monitor &&
cp -R "$tmp/skills/bionemo-kermt-monitor" ~/.cursor/skills/kermt-monitor &&
rm -rf "$tmp"
```

For one project only, put the folder in `.cursor/skills/kermt-monitor` inside it instead.

### Gemini CLI

Run in a terminal (add `--scope workspace` to install it for the current workspace only):

```sh
gemini skills install https://github.com/NVIDIA/skills.git --path skills/bionemo-kermt-monitor
```

Or copy the folder to `~/.gemini/skills/kermt-monitor` yourself.

### VS Code

For GitHub Copilot in VS Code, copy the skill folder to `~/.copilot/skills/kermt-monitor`:

```sh
tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/NVIDIA/skills.git "$tmp" &&
mkdir -p ~/.copilot/skills &&
rm -rf ~/.copilot/skills/kermt-monitor &&
cp -R "$tmp/skills/bionemo-kermt-monitor" ~/.copilot/skills/kermt-monitor &&
rm -rf "$tmp"
```

For one project only, put the folder in `.github/skills/kermt-monitor` inside it instead.

### Other agents

Get the folder:

```sh
git clone --depth 1 https://github.com/NVIDIA/skills.git
```

Then copy `skills/skills/bionemo-kermt-monitor` into your agent's skills folder as `kermt-monitor`. Agents that support SKILL.md read one folder per skill.

## FAQ

### What does the skill need to run?

It needs docker and jq, and a run directory containing run.json or a container name passed with --container.

### Does it change or stop my training run?

No. It only reads the manifest, logs and container state. It stops a container only if the user explicitly asks.

### Which workflows does it support?

The three pretrain workflows and kermt-finetune are covered directly, and other detached runs fall back to the most recent log file.

### Why might the container name be missing?

The run.json file does not yet store the launched container name, so the skill falls back to a filesystem lookup or needs --container.

### How do I install Kermt monitor in Claude Code?

Copy the skill folder from GitHub to ~/.claude/skills/kermt-monitor to use it in all your projects, or to .claude/skills/kermt-monitor in one project.

### Does Kermt monitor run scripts?

No. It is instructions only, with no scripts. This comes from a static check of the files on GitHub; nothing was run. Read the files before you install it.

### Which AI agents can use Kermt monitor?

Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Windsurf and Goose. Each reads skills in the SKILL.md format from its own folder; the install steps on this page list them.

### What are the best alternatives to Kermt monitor?

The closest alternatives to Kermt monitor are Huggingface trackio, Tao train rtdetr, Tao run deft pas, Tao run automl deft pipeline and Tao train dino.

## Where this comes from

- Description: from the skill's own SKILL.md.

## Alternatives

- [Huggingface trackio](https://ai-powered.com/skills/huggingface-trackio): Huggingface trackio is an open-source agent skill for ai models and llm tooling.
- [Tao train rtdetr](https://ai-powered.com/skills/tao-train-rtdetr): Tao train rtdetr is an open-source agent skill for ai models and llm tooling.
- [Tao run deft pas](https://ai-powered.com/skills/tao-run-deft-pas): Tao run deft pas is an open-source agent skill for ai models and llm tooling.
- [Tao run automl deft pipeline](https://ai-powered.com/skills/tao-run-automl-deft-pipeline): Tao run automl deft pipeline is an open-source agent skill for ai models and llm tooling.
- [Tao train dino](https://ai-powered.com/skills/tao-train-dino): Tao train dino is an open-source agent skill for ai models and llm tooling.
- [Tao train deformable detr](https://ai-powered.com/skills/tao-train-deformable-detr): Tao train deformable detr is an open-source agent skill for ai models and llm tooling.
- [Skill creator](https://ai-powered.com/skills/skill-creator): Skill creator is a free agent skill for ai models and llm tooling.
- [Nemo rl auto research](https://ai-powered.com/skills/nemo-rl-auto-research): Nemo rl auto research is an open-source agent skill for developer tools.

---
Source: https://ai-powered.com/skills/kermt-monitor
