
Open-source gateway routing apps to 250+ LLMs with retries, fallbacks and caching
- Open source
- 13.4K GitHub stars
- Developer tools

Open-source gateway routing apps to 250+ LLMs with retries, fallbacks and caching

Open-source AI gateway in Go that routes LLM calls with budgets, fallbacks and MCP
Rust gateway that proxies OpenAI and Anthropic APIs and records each generation
Self-hosted proxy that logs token use, cost and latency for LLM API calls and enforces key

Self-hosted AI gateway routing many providers through one local endpoint with fallback
OpenAI-compatible gateway to around 1,000 LLMs with routing, cost tracking and team
Self-hosted benchmark that scores local language models as agents on 40 tool-use tasks

Open-source gateway adding identity, policy and audit between AI agents and your systems

Ruby library with one API for 23 AI providers, tools and Rails conventions
The closest alternatives to ShanHaiGateway are gateway, bifrost, langfuse, Llmtee and OmniRoute.
Yes. gateway, bifrost, langfuse have a free or open-source option.