
Guides validating and enabling MoE expert-parallel communication overlap
- Open source
- 4K GitHub stars
- Developer tools


Guides validating and enabling MoE expert-parallel communication overlap

Guides enabling and verifying TP, DP and PP communication overlap in Megatron-Bridge

Guides an agent through evidence-gated MoE performance tuning in Megatron Bridge

Guides choice of MoE token dispatcher (alltoall, DeepEP, HybridEP) by hardware and EP size

Guides choosing and combining TP, DP, PP, CP and EP degrees for Megatron Bridge training

Hardware-specific MoE training playbooks and candidate parallelism configs for Megatron

Guides long-context MoE training in Megatron Bridge: CP sizing, recompute and dispatcher

Configures CPU offloading in Megatron Bridge to cut GPU memory during training

Guidance for choosing FSDP or 3D parallelism when training MoE vision-language models

Guides selective and full activation recompute in Megatron Bridge to cut GPU memory use

Guides an agent through reducing peak GPU memory in Megatron Bridge training runs

Runs Megatron-LM and Megatron Bridge training and checks their loss curves match
The closest alternatives to Nemo mbridge perf moe comm overlap are Nemo mbridge perf expert parallel overlap, Nemo mbridge perf tp dp comm overlap, Nemo mbridge perf moe optimization workflow, Nemo mbridge perf moe dispatcher selection and Nemo mbridge perf parallelism strategies.
Yes. Nemo mbridge perf expert parallel overlap, Nemo mbridge perf tp dp comm overlap, Nemo mbridge perf moe optimization workflow have a free or open-source option.