
Guides choosing and combining TP, DP, PP, CP and EP degrees for Megatron Bridge training
- Open source
- 4K GitHub stars
- AI models and LLM tooling


Guides choosing and combining TP, DP, PP, CP and EP degrees for Megatron Bridge training

Guides enabling and verifying TP, DP and PP communication overlap in Megatron-Bridge

Runs Megatron-LM and Megatron Bridge training and checks their loss curves match

Guides an agent through enabling Megatron FSDP sharded training in Megatron-Bridge

Guides long-context MoE training in Megatron Bridge: CP sizing, recompute and dispatcher

Picks Megatron Bridge training recipes and resizes parallelism for your GPU allocation

Guides an agent through reducing peak GPU memory in Megatron Bridge training runs

Hardware-specific MoE training playbooks and candidate parallelism configs for Megatron

Configures CPU offloading in Megatron Bridge to cut GPU memory during training

Guides tuning of MoE expert-parallel communication overlap in Megatron Bridge

Guides Megatron-Bridge sequence packing and long-context training setup and validation

Guides selective and full activation recompute in Megatron Bridge to cut GPU memory use
The closest alternatives to Nemo mbridge perf hierarchical context parallel are Nemo mbridge perf parallelism strategies, Nemo mbridge perf tp dp comm overlap, Nemo mbridge mlm bridge training, Nemo mbridge perf megatron fsdp and Nemo mbridge perf moe long context.
Yes. Nemo mbridge perf parallelism strategies, Nemo mbridge perf tp dp comm overlap, Nemo mbridge mlm bridge training have a free or open-source option.