
Jetson LLM serve
SkillGuides an agent through launching a vLLM or SGLang OpenAI-compatible server on Jetson
- Open source
- 4K GitHub stars
- AI models and LLM tooling


Guides an agent through launching a vLLM or SGLang OpenAI-compatible server on Jetson

Recommends an LLM serving runtime and memory flags for NVIDIA Jetson devices

Benchmarks LLM and VLM serving speed on Jetson devices with structured JSON results

Guides an agent through reducing peak GPU memory in Megatron Bridge training runs

Guides choice of MoE token dispatcher (alltoall, DeepEP, HybridEP) by hardware and EP size

Hardware-specific MoE training playbooks and candidate parallelism configs for Megatron

Guides an agent through training sentence-transformers embedding, reranker and sparse

Trains, distills, quantizes and runs RT-DETR real-time object detection models

Builds and fixes NVIDIA speech-to-speech voice agents on Pipecat or LiveKit

Guidance for choosing FSDP or 3D parallelism when training MoE vision-language models

Guides agents through self-critique, reflection loops and LLM-as-judge evaluation

Fine-tunes Parakeet TDT ASR on clinical vocabulary and measures the gain offline
The closest alternatives to Jetson speculative decoding are Jetson LLM serve, Jetson inference mem tune, Jetson LLM benchmark, Nemo mbridge perf memory tuning and Nemo mbridge perf moe dispatcher selection.
Yes. Jetson LLM serve, Jetson inference mem tune, Jetson LLM benchmark have a free or open-source option.