
Benchmark Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama with structured JSON output.
- Open source
- 4K GitHub stars
- AI models and LLM tooling

Benchmark Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama with structured JSON output.

Stand up vLLM or SGLang serving on Jetson, using upstream vLLM on Thor and Orin JetPack 7.2+, and NVIDIA-AI-IOT vLLM on older Orin.

Measure Jetson DRAM/NvMap usage and verify before/after memory reclamation with live audit data.

Reclaim DRAM by disabling unused subsystems across MB1 BCT, MB2 BCT, kernel reserved-memory, and SWIOTLB

Pick Jetson-compatible containers, vLLM runtime images, and Jetson AI Lab PyPI indexes; maps Orin SM 8.7 vs Thor SM 11.0 and JetPack-specifi

Use when you need to print Jetson BSP info (L4T version, board configs, rootfs state) from a Linux_for_Tegra root on the host PC

Use when you need to print Jetson device info (module model, L4T version, kernel, OS version, current power mode) from a running Jetson targ

Use to promote overlay files and built artifacts into the staged BSP image


Switch the active Jetson target-platform pointer to an existing profile YAML

Add EAGLE-3 or draft-model speculative decoding to a Jetson vLLM server when TPOT is the bottleneck.

Use after jetson-flash-image to run static BSP checks, on-target smoke/regression tests on a flashed DUT, or both

Use when measuring Jetson Video Codec SDK or PyNvVideoCodec encode/decode throughput, comparing presets or surfaces, testing codec-worker ca

Use when Jetson codec, profile, chroma, bit-depth, dimension, engine-count, or operational support must be reconciled from live APIs, authen

Use when planning, executing, and independently validating Jetson Video Codec SDK or PyNvVideoCodec encode/decode, transcode, segmentation,

Use when turning a Jetson encoder use case into one surface-neutral recipe with native Video Codec SDK and PyNvVideoCodec projections.

Use when installing, repairing, reusing, inspecting, or verifying readiness of the native NVIDIA Video Codec SDK or PyNvVideoCodec on Jetson

Playbook for launching, monitoring, stopping, and debugging NeMo-RL recipes on a Kubernetes cluster via the nrl-k8s CLI

Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.


How to launch distributed Megatron-LM training jobs on a SLURM cluster

Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.

Exports sanitized metadata, parameters, reproducibility details, quality metrics, and optional review artifacts from Medical AI inference ru

Guide for selecting and configuring distributed training strategies in NeMo AutoModel, including FSDP2, Megatron FSDP, DDP, and parallelism

Configure NeMo AutoModel job launches for interactive runs, Slurm clusters, and SkyPilot cloud execution.

Guide for onboarding new model architectures into NeMo AutoModel, including architecture discovery, implementation patterns, registration, a

Create and modify NeMo AutoModel training and evaluation recipes, including YAML structure, builders, and execution flow.

Build, migrate, review, and maintain third-party NVIDIA NeMo Fabric adapters against the public adapter contract

Use this skill when integrating NVIDIA NeMo Fabric into a consumer application, service, evaluation harness, or platform through the typed P

Run Megatron-LM (MLM) and Megatron Bridge training with mock or real data