# gateway

> gateway is an open-source AI tool for developer tools. Open-source gateway routing apps to 250+ LLMs with retries, fallbacks and caching. It is available on API, CLI, self-hosted and Web.

- **Kind:** Tool
- **Pricing:** Open source
- **Starting price:** $49 a month
- **Platforms:** API, CLI, self-hosted, Web
- **Website:** https://portkey.ai/features/ai-gateway
- **Repository:** https://github.com/portkey-ai/gateway
- **GitHub stars:** 13,399
- **License:** MIT
- **Categories:** [developer tools](https://ai-powered.com/categories/developer-tools)
- **Listing updated:** 2026-11-18

## About

The README describes this as a lightweight router that sits between your application and many model providers, so one OpenAI-style API can reach hundreds of language, vision, audio and image models. Requests can be retried, failed over to another provider, load balanced across keys, or served from a cache when the same question comes in again.

It runs locally with a single npx command, exposes a local endpoint and a console for viewing request logs, and can be deployed through Docker, Node.js, Cloudflare or Replit. Routing and guardrail behaviour is controlled by config objects passed with each client, so teams can change reliability rules without touching application code.

The project suits developers who call several model providers and want failover, cost tracking and observability without writing that plumbing themselves. Some features, such as semantic caching and prompt management, are noted in the README as available only in the hosted or enterprise versions.

## Features

- **Provider fallbacks:** Switches to another provider or model when a request fails with a chosen error type.
- **Automatic retries:** Retries failed requests up to five times using exponential backoff.
- **Load balancing:** Distributes requests across several API keys or providers using weights.
- **Simple caching:** Caches repeat LLM responses to lower cost and latency.
- **Configurable guardrails:** Checks inputs and outputs against rules, with pre-built and custom checks.
- **Request logging console:** Shows local request logs in a built-in console at the gateway address.

## Good for

- Keep an AI feature running when one model provider has an outage
- Compare or switch between many LLMs through one OpenAI-compatible API
- Cap retries and timeouts on slow or unreliable model endpoints
- Track request volume, latency and cost across several AI providers
- Add content guardrails to model replies before they reach users

## FAQ

### What license is the gateway released under?

The repository is licensed under MIT.

### How do I start it locally?

The README gives the command npx @portkey-ai/gateway, which runs the gateway on localhost port 8787 with a console at /public/.

### Which features are only in the hosted or enterprise versions?

The README marks semantic caching and prompt template management as available in the hosted and enterprise versions.

### Which agent frameworks does it work with?

The README lists Autogen, CrewAI, LangChain, LlamaIndex, Phidata and Control Flow, plus custom agents.

### Is gateway free?

Yes. gateway is open source, so you can use and self-host it for free.

### What are the best alternatives to gateway?

The closest alternatives to gateway are ShanHaiGateway, OmniRoute, bifrost, langfuse and Eden AI.

## Where this comes from

- Description: written from the project's README and the project's website.

## Alternatives

- [ShanHaiGateway](https://ai-powered.com/tools/shanhaigateway): ShanHaiGateway is an open-source AI tool for developer tools.
- [OmniRoute](https://ai-powered.com/tools/omniroute): OmniRoute is a freemium AI tool for ai models and llm tooling.
- [bifrost](https://ai-powered.com/tools/bifrost-3): bifrost is an open-source AI tool for ai models and llm tooling.
- [langfuse](https://ai-powered.com/tools/langfuse-2): langfuse is an open-source AI tool for developer tools.
- [Eden AI](https://ai-powered.com/tools/eden_ai): Eden AI is a paid AI tool for developer tools.
- [Llmtee](https://ai-powered.com/tools/llmtee): Llmtee is an open-source AI tool for developer tools.
- [ruby_llm](https://ai-powered.com/tools/ruby-llm): ruby_llm is an open-source AI tool for developer tools.
- [cc-switch](https://ai-powered.com/tools/cc-switch): cc-switch is an open-source AI tool for developer tools.

---
Source: https://ai-powered.com/tools/gateway-10
