
Guide for onboarding new model architectures into NeMo AutoModel, including architecture discovery, implementation patterns, registration, a
- Open source
- 4K GitHub stars

Guide for onboarding new model architectures into NeMo AutoModel, including architecture discovery, implementation patterns, registration, a

Create and modify NeMo AutoModel training and evaluation recipes, including YAML structure, builders, and execution flow.

Build, migrate, review, and maintain third-party NVIDIA NeMo Fabric adapters against the public adapter contract

Use this skill when integrating NVIDIA NeMo Fabric into a consumer application, service, evaluation harness, or platform through the typed P

Run Megatron-LM (MLM) and Megatron Bridge training with mock or real data

Convert single-node scripts to multi-node Slurm sbatch jobs and debug common multi-node failures

Validate and use selective and full activation recompute in Megatron Bridge to reduce GPU memory usage at the cost of extra compute

Validate and use CPU offloading in Megatron Bridge, including layer-level activation offloading and fractional optimizer state offloading wi

Validate and use CUDA graph capture in Megatron Bridge, including local full-iteration graphs and Transformer Engine scoped graphs for atten

Validate and use MoE expert-parallel communication overlap in Megatron-Bridge, including overlap_moe_expert_parallel_comm, delay_wgrad_compu

Operational guide for enabling hierarchical context parallelism in Megatron-Bridge, including config knobs, code anchors, pitfalls, and veri

Operational guide for enabling Megatron FSDP in Megatron-Bridge, including config knobs, code anchors, pitfalls, and verification.

Techniques for reducing peak GPU memory in Megatron Bridge — expandable segments, PEFT + SP input re-gather, parallelism resizing, activatio

MoE expert-parallel communication overlap in Megatron Bridge

Choose the right MoE token dispatcher (`alltoall`, DeepEP, or HybridEP) for the hardware, EP degree, and optimization stage

Representative, point-in-time MoE training playbooks by hardware and model family

Long-context MoE training guidance for Megatron Bridge

Evidence-gated workflow for MoE performance optimization in Megatron Bridge

Practical guidance for training MoE VLMs in Megatron Bridge

Operational guide for choosing and combining parallelism strategies in Megatron Bridge, including sizing rules, hardware topology mapping, a

Validate and use packed sequences and long-context training in Megatron-Bridge, including offline LLM packing, collate-time VLM packing, Ene

Operational guide for enabling TP, DP, and PP communication overlap in Megatron-Bridge, including config knobs, code anchors, pitfalls, and

Recommend and customize Megatron Bridge library and benchmark recipes for a user's model, GPU count, hardware, sequence length, and pretrain

Resiliency features in Megatron Bridge including fault tolerance, straggler detection, in-process restart, preemption, and re-run state mach

Use this skill when NeMo Relay is installed or imported but application-side runtime behavior is missing or incorrect, including load failur

Use this skill when first-time NeMo Relay users want help selecting and following an appropriate quick-start guide or verifying their first

Use this skill when choosing or running NeMo Relay installation for an agent runtime, application language, framework integration, or plugin

Use this skill when an application owns tool or LLM/provider call sites and needs to wrap them with NeMo Relay scopes and managed execution

Use this skill when concurrent requests, async tasks, threads, workers, goroutines, or agents need independent NeMo Relay scope stacks and c

Use this skill when adding NeMo Relay typed wrappers, domain types, or provider codecs while preserving JSON middleware semantics and caller