
Configures CPU offloading in Megatron Bridge to cut GPU memory during training
- Open source
- 4K GitHub stars
- AI models and LLM tooling

Configures CPU offloading in Megatron Bridge to cut GPU memory during training

Guides validating and configuring CUDA graph capture in Megatron Bridge training runs

Guides validating and enabling MoE expert-parallel communication overlap

Guides an agent through enabling hierarchical context parallelism in Megatron-Bridge

Guides an agent through enabling Megatron FSDP sharded training in Megatron-Bridge

Guides an agent through reducing peak GPU memory in Megatron Bridge training runs

Guides tuning of MoE expert-parallel communication overlap in Megatron Bridge

Guides choice of MoE token dispatcher (alltoall, DeepEP, HybridEP) by hardware and EP size

Hardware-specific MoE training playbooks and candidate parallelism configs for Megatron

Guides long-context MoE training in Megatron Bridge: CP sizing, recompute and dispatcher

Guides an agent through evidence-gated MoE performance tuning in Megatron Bridge

Guidance for choosing FSDP or 3D parallelism when training MoE vision-language models

Guides choosing and combining TP, DP, PP, CP and EP degrees for Megatron Bridge training

Guides Megatron-Bridge sequence packing and long-context training setup and validation

Guides enabling and verifying TP, DP and PP communication overlap in Megatron-Bridge

Picks Megatron Bridge training recipes and resizes parallelism for your GPU allocation

Configures fault tolerance, straggler detection and restarts for Megatron Bridge training

Diagnoses NeMo Relay runtime problems like load failures, missing events and scope errors

Guides first-time NeMo Relay users to a matching quick-start and verifies their first

Guides choosing, installing and verifying the NeMo Relay package for an agent or app

Wraps existing tool and LLM calls with NeMo Relay scopes, lifecycle events and middleware

Keeps NeMo Relay scope stacks separate across concurrent requests, tasks and agents

Guides agents in adding typed wrappers and provider codecs to NeMo Relay integrations

Guides an agent through renaming NeMo Flow code and docs to NeMo Relay across languages

Guides configuring and measuring NeMo Relay adaptive plugin telemetry, hints

Guides building reusable NeMo Relay runtime plugins with validation and rollback-safe

Guides choosing and configuring NeMo Relay 0.6/0.7 observability exporters and subscribers

Lets your agent ingest, index and query document collections with the NeMo Retriever CLI

Lets an agent search and ingest documents through the NeMo Retriever MCP server

Runs iterative NeMo-RL experiments with git branches and TSV logs as a research ledger