
Jetson LLM serve
SkillGuides an agent through launching a vLLM or SGLang OpenAI-compatible server on Jetson
- Open source
- 4K GitHub stars
- AI models and LLM tooling


Guides an agent through launching a vLLM or SGLang OpenAI-compatible server on Jetson

Adds EAGLE-3 or draft-model speculative decoding to a Jetson vLLM server

Benchmarks LLM and VLM serving speed on Jetson devices with structured JSON results

Plans and applies reversible changes to free Jetson memory by going headless

Guides an agent through reducing peak GPU memory in Megatron Bridge training runs

Runs TAO model checkpoints as local inference microservices with start, query and stop

Choose the right SageMaker serving container and current image URI for a model

Deploys and tunes LLM and AI inference servers on GKE with GPUs and TPUs

Fine-tunes and runs NVIDIA CLIP models for image-text retrieval and zero-shot

Fine-tunes HuggingFace vision, VLM and LLM models in an NVIDIA GPU container

Runs container-backed hyperparameter search for NVIDIA TAO networks via AutoMLRunner

Trains, distills, quantizes and runs RT-DETR real-time object detection models
The closest alternatives to Jetson inference mem tune are Jetson LLM serve, Jetson speculative decoding, Jetson LLM benchmark, Jetson headless mode and Nemo mbridge perf memory tuning.
Yes. Jetson LLM serve, Jetson speculative decoding, Jetson LLM benchmark have a free or open-source option.