
Diagnoses GKE TPU and GPU nodes stuck NotReady after kernel panics or hardware lockups
- Open source
- 24.4K GitHub stars
- Cloud and infrastructure


Diagnoses GKE TPU and GPU nodes stuck NotReady after kernel panics or hardware lockups

Guides an agent through diagnosing GKE nodes stuck in NotReady status

Diagnoses and mitigates GKE GPU and TPU node disruptions from host maintenance events

Diagnoses GKE TPU training throughput drops using ML Diagnostics and Cloud Monitoring

Guides agents through diagnosing GKE TPU multi-slice training hangs using MXLA analyzer

Diagnoses failing GKE workloads from pod status, events, logs and network checks

Diagnoses unexpected Compute Engine VM reboots, stops and preemptions from host events

Diagnoses GKE TPU node pool stockouts caused by unhealthy pools holding reservation slots

Monitors GKE TPU nodes, node pools and workloads using system metrics and PromQL

Diagnoses GKE TPU slice creation stalls caused by unstable Compute Engine MIGs

Diagnoses GKE TPU VMs stuck in Compute Engine REPAIRING state and routes recovery

Diagnoses and fixes GKE Workload Identity errors that stop Pods reaching Google Cloud APIs
The closest alternatives to Gke AI troubleshooting jobset interruption are Gke AI troubleshooting node unresponsive timeout, Gke node notready, Gke AI troubleshooting handle disruption GPU tpu, Gke AI troubleshooting tpu performance degradation and Gke AI troubleshooting tpu mxla hang.
Yes. Gke AI troubleshooting node unresponsive timeout, Gke node notready, Gke AI troubleshooting handle disruption GPU tpu have a free or open-source option.