​
What is an AI Gateway?

An AI Gateway (or LLM Gateway) is a centralized proxy for LLM-based applications within an organization. This setup enhances governance, management, and control for enterprises. By routing LLM requests through a centralized gateway rather than directly to LLM providers, you gain multiple benefits:
  1. Less vendor lock-in: Facilitates easier migrations between different LLM providers.
  2. Cost control: Manage and monitor expenses on a team-by-team basis.
  3. Rate limit control: Enforces request limits on a team-by-team basis.
  4. Retries & Caching: Improves performance and reliability of LLM calls.
  5. Analytics: Provides insights into usage patterns and operational metrics.

​
Aporia Guardrails & AI Gateways

Aporia Guardrails is a great fit for AI Gateways: every new LLM app automatically gets default out-of-the-box guardrails for hallucinations, inappropriate responses, prompt injections, data leakage, and more. If a specific team needs to customize guardrails for their project, they can log-in to the Aporia dashboard and edit the different policies. Specific integration examples: If you’re using an AI Gateway not listed here, please contact us at support@aporia.com. We’d be happy to add more examples!