Skip to content
Skill

Gke AI troubleshooting tpu vbar oom

Diagnoses vbar_control_agent OOMs and TPU v6e device failures on GKE clusters

Install
Scripts
shell
Platforms
Claude Code, Codex, Cursor
GitHub stars
24.4K
Last commit
2 days ago
License
Apache-2.0
Listing updated
1 day ago

Similar skills

About Gke AI troubleshooting tpu vbar oom

Gke AI troubleshooting tpu vbar oom is an open-source agent skill for cloud and infrastructure. It is available on Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Windsurf and Goose.

This skill guides an agent through troubleshooting crashes and memory errors on TPU v6e nodes in Google Kubernetes Engine. It focuses on vbar_control_agent segfaults, memory cgroup OOM messages in serial console logs, and checksum corruption reported by the tpu-device-plugin, which the skill links to race conditions during TPU device resets and frequent metrics polling.

The workflow starts by establishing a time window around the incident and then searches Cloud Logging with predefined filter templates. It checks serial console output for OOM kills tied to vbar_control_agent, looks for metric fetch failures in tpu-device-plugin logs, and inspects pods for custom TPU monitoring code that frequently queries host metrics. When custom collection is found, the agent is directed to recommend disabling it, and it can advise waiting for a future GKE version with a more resilient agent.

The skill expects gcloud access to the project and cluster, with Cloud Logging enabled, plus kubectl for listing pod images and environment details. Bash is used for the commands. It is scoped to TPU v6e issues and is not meant for general container OOM debugging.

When your agent uses it

Diagnoses and prevents vbar_control_agent segfaults, out-of-memory (OOM) errors, and TPU device initialization failures on TPU v6e nodes in GKE caused by race conditions during TPU device resets or high-frequency metrics polling. Use when troubleshooting vbar_control_agent crashes, memory cgroup OOMs in serial console logs, tpu-device-plugin metrics checksum corruption errors, or custom TPU metrics collection conflicts on GKE TPU v6e nodes. Don't use for general non-TPU container OOM troubleshooting or standard GKE node lifecycle operations.

From the skill's SKILL.md. Your agent reads this description to decide when to load the skill.

What it can access

From a static check of its 4 files (11 KB) on GitHub on 17 November 2026. Nothing was run. Read the files before you install a skill: it can tell your agent to run anything its tools allow.

Check another skill before you install it.

Install

Copy the skill folder to ~/.claude/skills/gke-ai-troubleshooting-tpu-vbar-oom to use it in all your projects:

tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.claude/skills &&
rm -rf ~/.claude/skills/gke-ai-troubleshooting-tpu-vbar-oom &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-vbar-oom" ~/.claude/skills/gke-ai-troubleshooting-tpu-vbar-oom &&
rm -rf "$tmp"

For one project only, put the folder in .claude/skills/gke-ai-troubleshooting-tpu-vbar-oom inside it instead.

Claude Code docs on skills

Copy the skill folder to ~/.agents/skills/gke-ai-troubleshooting-tpu-vbar-oom to use it in every repository:

tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.agents/skills &&
rm -rf ~/.agents/skills/gke-ai-troubleshooting-tpu-vbar-oom &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-vbar-oom" ~/.agents/skills/gke-ai-troubleshooting-tpu-vbar-oom &&
rm -rf "$tmp"

For one repository only, put the folder in .agents/skills/gke-ai-troubleshooting-tpu-vbar-oom inside it instead.

Codex docs on skills

Copy the skill folder to ~/.cursor/skills/gke-ai-troubleshooting-tpu-vbar-oom to use it in every project:

tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.cursor/skills &&
rm -rf ~/.cursor/skills/gke-ai-troubleshooting-tpu-vbar-oom &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-vbar-oom" ~/.cursor/skills/gke-ai-troubleshooting-tpu-vbar-oom &&
rm -rf "$tmp"

For one project only, put the folder in .cursor/skills/gke-ai-troubleshooting-tpu-vbar-oom inside it instead.

Cursor docs on skills

Run in a terminal (add --scope workspace to install it for the current workspace only):

gemini skills install https://github.com/google/skills.git --path skills/cloud/gke-ai-troubleshooting-tpu-vbar-oom

Or copy the folder to ~/.gemini/skills/gke-ai-troubleshooting-tpu-vbar-oom yourself.

Gemini CLI docs on skills

For GitHub Copilot in VS Code, copy the skill folder to ~/.copilot/skills/gke-ai-troubleshooting-tpu-vbar-oom:

tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.copilot/skills &&
rm -rf ~/.copilot/skills/gke-ai-troubleshooting-tpu-vbar-oom &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-vbar-oom" ~/.copilot/skills/gke-ai-troubleshooting-tpu-vbar-oom &&
rm -rf "$tmp"

For one project only, put the folder in .github/skills/gke-ai-troubleshooting-tpu-vbar-oom inside it instead.

VS Code docs on skills

Get the folder:

git clone --depth 1 https://github.com/google/skills.git

Then copy skills/skills/cloud/gke-ai-troubleshooting-tpu-vbar-oom into your agent's skills folder as gke-ai-troubleshooting-tpu-vbar-oom. Agents that support SKILL.md read one folder per skill.

The SKILL.md format

What it does

Serial console OOM detection
Filter templates locate memory cgroup OOM kills of vbar_control_agent in node serial logs.
Device plugin failure checks
Queries tpu-device-plugin error logs for metric checksum mismatches indicating vBAR corruption.
Custom metrics collector audit
Looks for workloads using libtpu monitoring that poll host metrics too often.
Time window calculation
Derives a 30-minute search window on either side of the reported incident time.
Mitigation guidance
Recommends disabling custom metrics collection and upgrading GKE once a fix ships.

Good for

  • Investigate why vbar_control_agent crashed on a TPU v6e node
  • Find out whether OOM kills appear in serial console logs for a node
  • Check GKE logs for tpu-device-plugin checksum corruption errors
  • Determine if a custom TPU metrics script is causing node instability

Gke AI troubleshooting tpu vbar oom FAQ

Which TPU hardware does this skill cover?

It targets TPU v6e nodes in GKE, as stated in its description.

What does it need to run?

Cloud Logging enabled for the project and access to the project and cluster through gcloud or an equivalent tool.

When should it not be used?

The description says not to use it for general non-TPU container OOM troubleshooting or standard GKE node lifecycle operations.

How do I install Gke AI troubleshooting tpu vbar oom in Claude Code?

Copy the skill folder from GitHub to ~/.claude/skills/gke-ai-troubleshooting-tpu-vbar-oom to use it in all your projects, or to .claude/skills/gke-ai-troubleshooting-tpu-vbar-oom in one project.

Does Gke AI troubleshooting tpu vbar oom run scripts?

Yes. It runs shell scripts. No network calls were found in its scripts. This comes from a static check of the files on GitHub; nothing was run. Read the files before you install it.

Which AI agents can use Gke AI troubleshooting tpu vbar oom?

Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Windsurf and Goose. Each reads skills in the SKILL.md format from its own folder; the install steps on this page list them.

What are the best alternatives to Gke AI troubleshooting tpu vbar oom?

The closest alternatives to Gke AI troubleshooting tpu vbar oom are Gke AI troubleshooting tpu performance degradation, Gke AI troubleshooting tpu metrics monitoring, Gke AI troubleshooting node unresponsive timeout, Gke AI troubleshooting tpu reservation stockout and Gke AI troubleshooting tpu vm repair stuck.

Where this comes from

  • Description: from the skill's own SKILL.md.

More from google/skills

  1. Route platform choices to Google's own skills by fetching and filtering its skill catalog

    • Open source
    • 24.4K GitHub stars
    • Cloud and infrastructure
  2. Guardrails for gcloud commands: validate syntax with help, scope projects, dry-run first

    • Open source
    • 24.4K GitHub stars
    • Cloud and infrastructure
  3. Authenticate to Google Cloud: ADC, impersonation, service accounts and IAM roles

    • Open source
    • 24.4K GitHub stars
    • Cloud and infrastructure
  4. First-time Google Cloud setup: sign in, pick a project, link billing, with consent gates

    • Open source
    • 24.4K GitHub stars
    • Cloud and infrastructure
  5. Look up Google developer docs through the Developer Knowledge MCP server or REST fallback

    • Open source
    • 24.4K GitHub stars
    • Developer tools