
Guides validating and enabling MoE expert-parallel communication overlap
- Open source
- 4K GitHub stars
- Developer tools


Guides validating and enabling MoE expert-parallel communication overlap

Guides tuning of MoE expert-parallel communication overlap in Megatron Bridge

Guides choosing and combining TP, DP, PP, CP and EP degrees for Megatron Bridge training

Guides an agent through enabling hierarchical context parallelism in Megatron-Bridge

Guides an agent through enabling Megatron FSDP sharded training in Megatron-Bridge

Runs Megatron-LM and Megatron Bridge training and checks their loss curves match

Guides an agent through reducing peak GPU memory in Megatron Bridge training runs

Guides an agent through evidence-gated MoE performance tuning in Megatron Bridge

Configures fault tolerance, straggler detection and restarts for Megatron Bridge training

Configures CPU offloading in Megatron Bridge to cut GPU memory during training

Guides validating and configuring CUDA graph capture in Megatron Bridge training runs

Picks Megatron Bridge training recipes and resizes parallelism for your GPU allocation
The closest alternatives to Nemo mbridge perf tp dp comm overlap are Nemo mbridge perf expert parallel overlap, Nemo mbridge perf moe comm overlap, Nemo mbridge perf parallelism strategies, Nemo mbridge perf hierarchical context parallel and Nemo mbridge perf megatron fsdp.
Yes. Nemo mbridge perf expert parallel overlap, Nemo mbridge perf moe comm overlap, Nemo mbridge perf parallelism strategies have a free or open-source option.