
Guides validating and enabling MoE expert-parallel communication overlap
- Open source
- 4K GitHub stars
- Developer tools


Guides validating and enabling MoE expert-parallel communication overlap

Guides validating and configuring CUDA graph capture in Megatron Bridge training runs

Guides an agent through reducing peak GPU memory in Megatron Bridge training runs

Hardware-specific MoE training playbooks and candidate parallelism configs for Megatron

Guides tuning of MoE expert-parallel communication overlap in Megatron Bridge

Guides choice of MoE token dispatcher (alltoall, DeepEP, HybridEP) by hardware and EP size

Guides an agent through multi-node Slurm sbatch jobs and debugging their failures

Configures fault tolerance, straggler detection and restarts for Megatron Bridge training

Guides enabling and verifying TP, DP and PP communication overlap in Megatron-Bridge

Guides configuring and measuring NeMo Relay adaptive plugin telemetry, hints

Practitioner runbook for training and serving large models, from GPUs to NaN loss spikes

Guides an agent through iterative profiling and tuning of cuTile GPU kernels in TileGym
The closest alternatives to Nemo mbridge perf moe optimization workflow are Nemo mbridge perf expert parallel overlap, Nemo mbridge perf cuda graphs, Nemo mbridge perf memory tuning, Nemo mbridge perf moe hardware configs and Nemo mbridge perf moe comm overlap.
Yes. Nemo mbridge perf expert parallel overlap, Nemo mbridge perf cuda graphs, Nemo mbridge perf memory tuning have a free or open-source option.