Skip to content

gateway vs ShanHaiGateway

gateway is an open-source AI tool for developer tools. ShanHaiGateway is an open-source AI tool for developer tools.

  • gateway is best for keep an ai feature running when one model provider has an outage.
  • ShanHaiGateway is best for spreading traffic across several model provider accounts behind one base url.
What it is
gateway is an open-source AI tool for developer tools.
ShanHaiGateway is an open-source AI tool for developer tools.
Best for
Keep an AI feature running when one model provider has an outage
Spreading traffic across several model provider accounts behind one base URL
Kind
Tool
Tool
Pricing
Open source, from $49 a month
Open source
Platforms
API, CLI, self-hosted, Web
API, CLI, self-hosted, Linux, macOS, Windows
GitHub stars
13.4K
Last commit
May 2026
License
MIT
Open source
Good for
Keep an AI feature running when one model provider has an outage; Compare or switch between many LLMs through one OpenAI-compatible API; Cap retries and timeouts on slow or unreliable model endpoints
Spreading traffic across several model provider accounts behind one base URL; Keeping multi-turn agent sessions on one upstream to improve prompt cache hit rates; Automatically sidelining a provider whose quota or model has been withdrawn

What gateway does

  • Provider fallbacks

    Switches to another provider or model when a request fails with a chosen error type.

  • Automatic retries

    Retries failed requests up to five times using exponential backoff.

  • Load balancing

    Distributes requests across several API keys or providers using weights.

  • Simple caching

    Caches repeat LLM responses to lower cost and latency.

  • Configurable guardrails

    Checks inputs and outputs against rules, with pre-built and custom checks.

  • Request logging console

    Shows local request logs in a built-in console at the gateway address.

What ShanHaiGateway does

  • Weighted round-robin routing

    Distributes requests across backends by configurable weight, concurrency and timeout, with a separate fallback chain.

  • Error-class based cooldown

    Applies exponential backoff for quota and permission errors and queues requests at capacity instead of returning 503.

  • Job-level session affinity

    Keeps requests from the same job or session on one backend to reuse provider-side prefix caches.

  • Hot-reloaded configuration

    Reloads backends.json when its modification time changes, so weight changes and new backends apply without a restart.

  • Quota-window sidecar

    A standard-library Python process that polls quota windows and temporarily removes exhausted backends.

  • Docker deployment

    Multi-stage build with a pure-Go SQLite driver, non-root runtime and a health endpoint.

gateway vs ShanHaiGateway: questions

Is gateway or ShanHaiGateway cheaper?

Both are open source. Compare current plans on each official site.

What is the difference between gateway and ShanHaiGateway?

gateway is an open-source AI tool for developer tools. ShanHaiGateway is an open-source AI tool for developer tools.

Keep comparing