
Diagnoses GKE TPU training throughput drops using ML Diagnostics and Cloud Monitoring
- Open source
- 24.4K GitHub stars
- Cloud and infrastructure


Diagnoses GKE TPU training throughput drops using ML Diagnostics and Cloud Monitoring

Monitors GKE TPU nodes, node pools and workloads using system metrics and PromQL

Diagnoses GKE TPU and GPU nodes stuck NotReady after kernel panics or hardware lockups

Diagnoses GKE TPU node pool stockouts caused by unhealthy pools holding reservation slots

Diagnoses GKE TPU VMs stuck in Compute Engine REPAIRING state and routes recovery

Guides agents through diagnosing GKE TPU multi-slice training hangs using MXLA analyzer

Diagnoses and mitigates GKE GPU and TPU node disruptions from host maintenance events

Diagnoses GKE TPU slice creation stalls caused by unstable Compute Engine MIGs

Diagnoses GKE TPU SliceCreationFailed errors caused by overlapping partitions or nodes

Diagnoses failing GKE workloads from pod status, events, logs and network checks

Inspect, troubleshoot and validate GKE TPU dynamic slice resources and workloads

Diagnoses GKE persistent storage failures like attach, mount, pressure and expansion
The closest alternatives to Gke AI troubleshooting tpu vbar oom are Gke AI troubleshooting tpu performance degradation, Gke AI troubleshooting tpu metrics monitoring, Gke AI troubleshooting node unresponsive timeout, Gke AI troubleshooting tpu reservation stockout and Gke AI troubleshooting tpu vm repair stuck.
Yes. Gke AI troubleshooting tpu performance degradation, Gke AI troubleshooting tpu metrics monitoring, Gke AI troubleshooting node unresponsive timeout have a free or open-source option.