gateway vs ShanHaiGateway
gateway is an open-source AI tool for developer tools. ShanHaiGateway is an open-source AI tool for developer tools.
- gateway is best for keep an ai feature running when one model provider has an outage.
- ShanHaiGateway is best for spreading traffic across several model provider accounts behind one base url.
- What it is
- gateway is an open-source AI tool for developer tools.ShanHaiGateway is an open-source AI tool for developer tools.
- Best for
- Keep an AI feature running when one model provider has an outageSpreading traffic across several model provider accounts behind one base URL
- Kind
- ToolTool
- Pricing
- Open source, from $49 a monthOpen source
- Platforms
- API, CLI, self-hosted, WebAPI, CLI, self-hosted, Linux, macOS, Windows
- GitHub stars
- 13.4K
- Last commit
- May 2026
- License
- MIT
- Open source
- Good for
- Keep an AI feature running when one model provider has an outage; Compare or switch between many LLMs through one OpenAI-compatible API; Cap retries and timeouts on slow or unreliable model endpointsSpreading traffic across several model provider accounts behind one base URL; Keeping multi-turn agent sessions on one upstream to improve prompt cache hit rates; Automatically sidelining a provider whose quota or model has been withdrawn
What gateway does
Provider fallbacks
Switches to another provider or model when a request fails with a chosen error type.
Automatic retries
Retries failed requests up to five times using exponential backoff.
Load balancing
Distributes requests across several API keys or providers using weights.
Simple caching
Caches repeat LLM responses to lower cost and latency.
Configurable guardrails
Checks inputs and outputs against rules, with pre-built and custom checks.
Request logging console
Shows local request logs in a built-in console at the gateway address.
What ShanHaiGateway does
Weighted round-robin routing
Distributes requests across backends by configurable weight, concurrency and timeout, with a separate fallback chain.
Error-class based cooldown
Applies exponential backoff for quota and permission errors and queues requests at capacity instead of returning 503.
Job-level session affinity
Keeps requests from the same job or session on one backend to reuse provider-side prefix caches.
Hot-reloaded configuration
Reloads backends.json when its modification time changes, so weight changes and new backends apply without a restart.
Quota-window sidecar
A standard-library Python process that polls quota windows and temporarily removes exhausted backends.
Docker deployment
Multi-stage build with a pure-Go SQLite driver, non-root runtime and a health endpoint.
gateway vs ShanHaiGateway: questions
Is gateway or ShanHaiGateway cheaper?
Both are open source. Compare current plans on each official site.
What is the difference between gateway and ShanHaiGateway?
gateway is an open-source AI tool for developer tools. ShanHaiGateway is an open-source AI tool for developer tools.
