# Gke AI troubleshooting tpu reservation stockout

> Gke AI troubleshooting tpu reservation stockout is an open-source agent skill for cloud and infrastructure. Diagnoses GKE TPU node pool stockouts caused by unhealthy pools holding reservation slots. It is available on Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Windsurf and Goose.

- **Kind:** Skill
- **Pricing:** Open source
- **Platforms:** Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Windsurf, Goose
- **Website:** https://github.com/google/skills/tree/main/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout
- **Repository:** https://github.com/google/skills
- **GitHub stars:** 24,379
- **License:** Apache-2.0
- **Skill name:** gke-ai-troubleshooting-tpu-reservation-stockout
- **Collection:** https://github.com/google/skills
- **Categories:** [cloud and infrastructure](https://ai-powered.com/categories/cloud-and-infrastructure)
- **Listing updated:** 2026-11-18

## About

This skill guides an agent through finding out why a GKE TPU node pool cannot be created or scaled up even though a specific Compute Engine TPU reservation exists. It checks Cloud Audit Logs and GKE operations for stockout errors, then compares reservation usage against the free slots the new pool needs. When the slots are held by VMs in a NotReady node pool with auto-repair disabled, it identifies that pool and walks through the checks needed to confirm it.

The workflow is read-only for diagnosis. It lists reservation consumers with gcloud, reads node readiness from Cloud Monitoring, and uses kubectl to map TPU nodes to their node pools. The skill explicitly warns against deleting healthy pools or changing instance groups directly, and it points the user to GKE documentation for any fix they choose to apply themselves.

The skill expects gcloud, kubectl and the relevant IAM viewer roles on the cluster, reservation and logging projects. It applies to Standard clusters only, since Autopilot always auto-repairs nodes. Shared reservations require the owner project to be known.

## When your agent uses it

> Diagnose and resolve cluster-wide GKE TPU node pool creation or scale-up failures caused by Compute Engine reservation stockouts (GCE_STOCKOUT, ZONE_RESOURCE_POOL_EXHAUSTED) when existing unhealthy or NotReady TPU node pools in the cluster have node auto-repair disabled and hold reserved TPU capacity. Use when TPU node pool creation or scale-up fails with GCE_STOCKOUT, ZONE_RESOURCE_POOL_EXHAUSTED, or ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS despite having a specific Compute Engine TPU reservation. Don't use for general pod scheduling without a TPU reservation or workload-level container crashes.

## What it can access

From a static check of the files on GitHub; nothing was run.

- [Instructions only, no scripts](https://github.com/google/skills/tree/HEAD/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout)
- [License: Apache-2.0](https://github.com/google/skills/tree/HEAD/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout)

## Features

- **Stockout error confirmation:** Searches logs and node pool operations for GCE_STOCKOUT and related capacity errors.
- **Reservation utilization check:** Compares reserved instance counts with in-use counts to measure free TPU slots.
- **Leaking node pool detection:** Finds NotReady TPU node pools whose VMs still occupy reservation capacity.
- **Auto-repair verification:** Checks whether node auto-repair is enabled on the suspect node pool and reviews repair events.
- **Guided remediation steps:** Explains which fixes free capacity and links GKE documentation for each option.

## Good for

- Why does my GKE TPU node pool fail to create with ZONE_RESOURCE_POOL_EXHAUSTED?
- Find which VMs are holding my TPU reservation slots
- Check whether an unhealthy TPU node pool is blocking scale-up
- Verify auto-repair settings on a Standard GKE TPU node pool
- Confirm enough reservation slots are free before retrying a node pool

## Install

### Claude Code

Copy the skill folder to `~/.claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout` to use it in all your projects:

```sh
tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.claude/skills &&
rm -rf ~/.claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout" ~/.claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
rm -rf "$tmp"
```

For one project only, put the folder in `.claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout` inside it instead.

### Codex

Copy the skill folder to `~/.agents/skills/gke-ai-troubleshooting-tpu-reservation-stockout` to use it in every repository:

```sh
tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.agents/skills &&
rm -rf ~/.agents/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout" ~/.agents/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
rm -rf "$tmp"
```

For one repository only, put the folder in `.agents/skills/gke-ai-troubleshooting-tpu-reservation-stockout` inside it instead.

### Cursor

Copy the skill folder to `~/.cursor/skills/gke-ai-troubleshooting-tpu-reservation-stockout` to use it in every project:

```sh
tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.cursor/skills &&
rm -rf ~/.cursor/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout" ~/.cursor/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
rm -rf "$tmp"
```

For one project only, put the folder in `.cursor/skills/gke-ai-troubleshooting-tpu-reservation-stockout` inside it instead.

### Gemini CLI

Run in a terminal (add `--scope workspace` to install it for the current workspace only):

```sh
gemini skills install https://github.com/google/skills.git --path skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout
```

Or copy the folder to `~/.gemini/skills/gke-ai-troubleshooting-tpu-reservation-stockout` yourself.

### VS Code

For GitHub Copilot in VS Code, copy the skill folder to `~/.copilot/skills/gke-ai-troubleshooting-tpu-reservation-stockout`:

```sh
tmp=$(mktemp -d) &&
git clone --depth 1 https://github.com/google/skills.git "$tmp" &&
mkdir -p ~/.copilot/skills &&
rm -rf ~/.copilot/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
cp -R "$tmp/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout" ~/.copilot/skills/gke-ai-troubleshooting-tpu-reservation-stockout &&
rm -rf "$tmp"
```

For one project only, put the folder in `.github/skills/gke-ai-troubleshooting-tpu-reservation-stockout` inside it instead.

### Other agents

Get the folder:

```sh
git clone --depth 1 https://github.com/google/skills.git
```

Then copy `skills/skills/cloud/gke-ai-troubleshooting-tpu-reservation-stockout` into your agent's skills folder as `gke-ai-troubleshooting-tpu-reservation-stockout`. Agents that support SKILL.md read one folder per skill.

## FAQ

### When should this skill be used?

When TPU node pool creation or scale-up fails with GCE_STOCKOUT, ZONE_RESOURCE_POOL_EXHAUSTED, or the related lack-of-capacity message despite having a specific Compute Engine TPU reservation.

### Does the skill change my cluster?

No. Diagnostic steps are read-only, and the skill tells the agent to never drain, delete or re-create nodes. Any fix is given to the user to apply.

### Does it apply to Autopilot clusters?

No. The skill applies to Standard clusters, because Autopilot clusters always auto-repair nodes and this cannot be turned off.

### Will enabling auto-repair free reservation slots?

According to the skill, no. Re-created TPU slice VMs still consume the reservation, so the blocked node pool still cannot get its slots until the unhealthy pool is deleted.

### How do I install Gke AI troubleshooting tpu reservation stockout in Claude Code?

Copy the skill folder from GitHub to ~/.claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout to use it in all your projects, or to .claude/skills/gke-ai-troubleshooting-tpu-reservation-stockout in one project.

### Does Gke AI troubleshooting tpu reservation stockout run scripts?

No. It is instructions only, with no scripts. This comes from a static check of the files on GitHub; nothing was run. Read the files before you install it.

### Which AI agents can use Gke AI troubleshooting tpu reservation stockout?

Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Windsurf and Goose. Each reads skills in the SKILL.md format from its own folder; the install steps on this page list them.

### What are the best alternatives to Gke AI troubleshooting tpu reservation stockout?

The closest alternatives to Gke AI troubleshooting tpu reservation stockout are Gke AI troubleshooting tpu vm repair stuck, Gke AI troubleshooting node unresponsive timeout, Gke AI troubleshooting tpu metrics monitoring, Gke AI troubleshooting handle disruption GPU tpu and Gke AI troubleshooting tpu slice mig stuck updating.

## Where this comes from

- Description: from the skill's own SKILL.md.

## Alternatives

- [Gke AI troubleshooting tpu vm repair stuck](https://ai-powered.com/skills/gke-ai-troubleshooting-tpu-vm-repair-stuck): Gke AI troubleshooting tpu vm repair stuck is an open-source agent skill for cloud and infrastructure.
- [Gke AI troubleshooting node unresponsive timeout](https://ai-powered.com/skills/gke-ai-troubleshooting-node-unresponsive-timeout): Gke AI troubleshooting node unresponsive timeout is an open-source agent skill for cloud and infrastructure.
- [Gke AI troubleshooting tpu metrics monitoring](https://ai-powered.com/skills/gke-ai-troubleshooting-tpu-metrics-monitoring): Gke AI troubleshooting tpu metrics monitoring is an open-source agent skill for cloud and infrastructure.
- [Gke AI troubleshooting handle disruption GPU tpu](https://ai-powered.com/skills/gke-ai-troubleshooting-handle-disruption-gpu-tpu): Gke AI troubleshooting handle disruption GPU tpu is an open-source agent skill for cloud and infrastructure.
- [Gke AI troubleshooting tpu slice mig stuck updating](https://ai-powered.com/skills/gke-ai-troubleshooting-tpu-slice-mig-stuck-updating): Gke AI troubleshooting tpu slice mig stuck updating is an open-source agent skill for cloud and infrastructure.
- [Gke AI troubleshooting tpu performance degradation](https://ai-powered.com/skills/gke-ai-troubleshooting-tpu-performance-degradation): Gke AI troubleshooting tpu performance degradation is an open-source agent skill for cloud and infrastructure.
- [Gke AI troubleshooting tpu partition overlap](https://ai-powered.com/skills/gke-ai-troubleshooting-tpu-partition-overlap): Gke AI troubleshooting tpu partition overlap is an open-source agent skill for cloud and infrastructure.
- [Aks GPU inference](https://ai-powered.com/skills/aks-gpu-inference): Aks GPU inference is an open-source agent skill for cloud and infrastructure.

---
Source: https://ai-powered.com/skills/gke-ai-troubleshooting-tpu-reservation-stockout
