
Guides choosing and combining TP, DP, PP, CP and EP degrees for Megatron Bridge training
- Open source
- 4K GitHub stars
- AI models and LLM tooling


Guides choosing and combining TP, DP, PP, CP and EP degrees for Megatron Bridge training

Hardware-specific MoE training playbooks and candidate parallelism configs for Megatron

Guidance for choosing FSDP or 3D parallelism when training MoE vision-language models

Guides selective and full activation recompute in Megatron Bridge to cut GPU memory use

Guides tuning of MoE expert-parallel communication overlap in Megatron Bridge

Guides choice of MoE token dispatcher (alltoall, DeepEP, HybridEP) by hardware and EP size

Guides Megatron-Bridge sequence packing and long-context training setup and validation

Guides an agent through enabling hierarchical context parallelism in Megatron-Bridge

Guides an agent through reducing peak GPU memory in Megatron Bridge training runs

Configures CPU offloading in Megatron Bridge to cut GPU memory during training

Picks Megatron Bridge training recipes and resizes parallelism for your GPU allocation

Guides choosing and configuring FSDP2, Megatron FSDP, DDP and parallelism settings in NeMo
The closest alternatives to Nemo mbridge perf moe long context are Nemo mbridge perf parallelism strategies, Nemo mbridge perf moe hardware configs, Nemo mbridge perf moe vlm training, Nemo mbridge perf activation recompute and Nemo mbridge perf moe comm overlap.
Yes. Nemo mbridge perf parallelism strategies, Nemo mbridge perf moe hardware configs, Nemo mbridge perf moe vlm training have a free or open-source option.