
Picks Megatron Bridge training recipes and resizes parallelism for your GPU allocation
- Open source
- 4K GitHub stars
- AI models and LLM tooling


Picks Megatron Bridge training recipes and resizes parallelism for your GPU allocation

Guides long-context MoE training in Megatron Bridge: CP sizing, recompute and dispatcher

Guidance for choosing FSDP or 3D parallelism when training MoE vision-language models

Hardware-specific MoE training playbooks and candidate parallelism configs for Megatron

Guides tuning of MoE expert-parallel communication overlap in Megatron Bridge

Guides enabling and verifying TP, DP and PP communication overlap in Megatron-Bridge

Runs Megatron-LM and Megatron Bridge training and checks their loss curves match

Guides an agent through enabling hierarchical context parallelism in Megatron-Bridge

Guides choosing and configuring FSDP2, Megatron FSDP, DDP and parallelism settings in NeMo

Configures CPU offloading in Megatron Bridge to cut GPU memory during training

Guides an agent through enabling Megatron FSDP sharded training in Megatron-Bridge

Guides an agent through reducing peak GPU memory in Megatron Bridge training runs
The closest alternatives to Nemo mbridge perf parallelism strategies are Nemo mbridge recipe recommender, Nemo mbridge perf moe long context, Nemo mbridge perf moe vlm training, Nemo mbridge perf moe hardware configs and Nemo mbridge perf moe comm overlap.
Yes. Nemo mbridge recipe recommender, Nemo mbridge perf moe long context, Nemo mbridge perf moe vlm training have a free or open-source option.