
Hardware-specific MoE training playbooks and candidate parallelism configs for Megatron
- Open source
- 4K GitHub stars
- AI models and LLM tooling


Hardware-specific MoE training playbooks and candidate parallelism configs for Megatron

Guides validating and enabling MoE expert-parallel communication overlap

Guides an agent through evidence-gated MoE performance tuning in Megatron Bridge

Guides tuning of MoE expert-parallel communication overlap in Megatron Bridge

Guides long-context MoE training in Megatron Bridge: CP sizing, recompute and dispatcher

Guidance for choosing FSDP or 3D parallelism when training MoE vision-language models

Guides choosing and configuring FSDP2, Megatron FSDP, DDP and parallelism settings in NeMo

Guides an agent through reducing peak GPU memory in Megatron Bridge training runs

Guides choosing and combining TP, DP, PP, CP and EP degrees for Megatron Bridge training

Guides an agent through multi-node Slurm sbatch jobs and debugging their failures

Guides selective and full activation recompute in Megatron Bridge to cut GPU memory use

Picks Megatron Bridge training recipes and resizes parallelism for your GPU allocation
The closest alternatives to Nemo mbridge perf moe dispatcher selection are Nemo mbridge perf moe hardware configs, Nemo mbridge perf expert parallel overlap, Nemo mbridge perf moe optimization workflow, Nemo mbridge perf moe comm overlap and Nemo mbridge perf moe long context.
Yes. Nemo mbridge perf moe hardware configs, Nemo mbridge perf expert parallel overlap, Nemo mbridge perf moe optimization workflow have a free or open-source option.