Skip to content
Skill

Gke AI troubleshooting tpu reservation stockout

Diagnoses GKE TPU node pool stockouts caused by unhealthy pools holding reservation slots

Install
Scripts
None
Platforms
Claude Code, Codex, Cursor
GitHub stars
24.4K
Last commit
2 days ago
License
Apache-2.0
Listing updated
1 day ago

Similar skills

About Gke AI troubleshooting tpu reservation stockout

Gke AI troubleshooting tpu reservation stockout is an open-source agent skill for cloud and infrastructure. It is available on Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Windsurf and Goose.

This skill guides an agent through finding out why a GKE TPU node pool cannot be created or scaled up even though a specific Compute Engine TPU reservation exists. It checks Cloud Audit Logs and GKE operations for stockout errors, then compares reservation usage against the free slots the new pool needs. When the slots are held by VMs in a NotReady node pool with auto-repair disabled, it identifies that pool and walks through the checks needed to confirm it.

The workflow is read-only for diagnosis. It lists reservation consumers with gcloud, reads node readiness from Cloud Monitoring, and uses kubectl to map TPU nodes to their node pools. The skill explicitly warns against deleting healthy pools or changing instance groups directly, and it points the user to GKE documentation for any fix they choose to apply themselves.

The skill expects gcloud, kubectl and the relevant IAM viewer roles on the cluster, reservation and logging projects. It applies to Standard clusters only, since Autopilot always auto-repairs nodes. Shared reservations require the owner project to be known.

When your agent uses it

Diagnose and resolve cluster-wide GKE TPU node pool creation or scale-up failures caused by Compute Engine reservation stockouts (GCE_STOCKOUT, ZONE_RESOURCE_POOL_EXHAUSTED) when existing unhealthy or NotReady TPU node pools in the cluster have node auto-repair disabled and hold reserved TPU capacity. Use when TPU node pool creation or scale-up fails with GCE_STOCKOUT, ZONE_RESOURCE_POOL_EXHAUSTED, or ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS despite having a specific Compute Engine TPU reservation. Don't use for general pod scheduling without a TPU reservation or workload-level container crashes.

From the skill's SKILL.md. Your agent reads this description to decide when to load the skill.

What it can access

From a static check of its 2 files (19 KB) on GitHub on 17 November 2026. Nothing was run. Read the files before you install a skill: it can tell your agent to run anything its tools allow.

Check another skill before you install it.

Install

Copy the skill folder to ~/.claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout to use it in all your projects:

tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.claude/skills &&
rm -rf ~/.claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout" ~/.claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
rm -rf "$tmp"

For one project only, put the folder in .claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout inside it instead.

Claude Code docs on skills

Copy the skill folder to ~/.agents/skills/gke-ai-troubleshooting-tpu-reservation-stockout to use it in every repository:

tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.agents/skills &&
rm -rf ~/.agents/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout" ~/.agents/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
rm -rf "$tmp"

For one repository only, put the folder in .agents/skills/gke-ai-troubleshooting-tpu-reservation-stockout inside it instead.

Codex docs on skills

Copy the skill folder to ~/.cursor/skills/gke-ai-troubleshooting-tpu-reservation-stockout to use it in every project:

tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.cursor/skills &&
rm -rf ~/.cursor/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout" ~/.cursor/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
rm -rf "$tmp"

For one project only, put the folder in .cursor/skills/gke-ai-troubleshooting-tpu-reservation-stockout inside it instead.

Cursor docs on skills

Run in a terminal (add --scope workspace to install it for the current workspace only):

gemini skills install https://github.com/google/skills.git --path skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout

Or copy the folder to ~/.gemini/skills/gke-ai-troubleshooting-tpu-reservation-stockout yourself.

Gemini CLI docs on skills

For GitHub Copilot in VS Code, copy the skill folder to ~/.copilot/skills/gke-ai-troubleshooting-tpu-reservation-stockout:

tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.copilot/skills &&
rm -rf ~/.copilot/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout" ~/.copilot/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
rm -rf "$tmp"

For one project only, put the folder in .github/skills/gke-ai-troubleshooting-tpu-reservation-stockout inside it instead.

VS Code docs on skills

Get the folder:

git clone --depth 1 https://github.com/google/skills.git

Then copy skills/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout into your agent's skills folder as gke-ai-troubleshooting-tpu-reservation-stockout. Agents that support SKILL.md read one folder per skill.

The SKILL.md format

What it does

Stockout error confirmation
Searches logs and node pool operations for GCE_STOCKOUT and related capacity errors.
Reservation utilization check
Compares reserved instance counts with in-use counts to measure free TPU slots.
Leaking node pool detection
Finds NotReady TPU node pools whose VMs still occupy reservation capacity.
Auto-repair verification
Checks whether node auto-repair is enabled on the suspect node pool and reviews repair events.
Guided remediation steps
Explains which fixes free capacity and links GKE documentation for each option.

Good for

  • Why does my GKE TPU node pool fail to create with ZONE_RESOURCE_POOL_EXHAUSTED?
  • Find which VMs are holding my TPU reservation slots
  • Check whether an unhealthy TPU node pool is blocking scale-up
  • Verify auto-repair settings on a Standard GKE TPU node pool
  • Confirm enough reservation slots are free before retrying a node pool

Gke AI troubleshooting tpu reservation stockout FAQ

When should this skill be used?

When TPU node pool creation or scale-up fails with GCE_STOCKOUT, ZONE_RESOURCE_POOL_EXHAUSTED, or the related lack-of-capacity message despite having a specific Compute Engine TPU reservation.

Does the skill change my cluster?

No. Diagnostic steps are read-only, and the skill tells the agent to never drain, delete or re-create nodes. Any fix is given to the user to apply.

Does it apply to Autopilot clusters?

No. The skill applies to Standard clusters, because Autopilot clusters always auto-repair nodes and this cannot be turned off.

Will enabling auto-repair free reservation slots?

According to the skill, no. Re-created TPU slice VMs still consume the reservation, so the blocked node pool still cannot get its slots until the unhealthy pool is deleted.

How do I install Gke AI troubleshooting tpu reservation stockout in Claude Code?

Copy the skill folder from GitHub to ~/.claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout to use it in all your projects, or to .claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout in one project.

Does Gke AI troubleshooting tpu reservation stockout run scripts?

No. It is instructions only, with no scripts. This comes from a static check of the files on GitHub; nothing was run. Read the files before you install it.

Which AI agents can use Gke AI troubleshooting tpu reservation stockout?

Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Windsurf and Goose. Each reads skills in the SKILL.md format from its own folder; the install steps on this page list them.

What are the best alternatives to Gke AI troubleshooting tpu reservation stockout?

The closest alternatives to Gke AI troubleshooting tpu reservation stockout are Gke AI troubleshooting tpu vm repair stuck, Gke AI troubleshooting node unresponsive timeout, Gke AI troubleshooting tpu metrics monitoring, Gke AI troubleshooting handle disruption GPU tpu and Gke AI troubleshooting tpu slice mig stuck updating.

Where this comes from

  • Description: from the skill's own SKILL.md.

More from google/skills

  1. Route platform choices to Google's own skills by fetching and filtering its skill catalog

    • Open source
    • 24.4K GitHub stars
    • Cloud and infrastructure
  2. Guardrails for gcloud commands: validate syntax with help, scope projects, dry-run first

    • Open source
    • 24.4K GitHub stars
    • Cloud and infrastructure
  3. Authenticate to Google Cloud: ADC, impersonation, service accounts and IAM roles

    • Open source
    • 24.4K GitHub stars
    • Cloud and infrastructure
  4. First-time Google Cloud setup: sign in, pick a project, link billing, with consent gates

    • Open source
    • 24.4K GitHub stars
    • Cloud and infrastructure
  5. Look up Google developer docs through the Developer Knowledge MCP server or REST fallback

    • Open source
    • 24.4K GitHub stars
    • Developer tools