Best LLM API Gateways for Auth, Logs, and Egress

Cloudflare’s AI Gateway is the edge leader on this list. One line of code, analytics, caching, rate limits, retries, and model fallback. Core features are free on every plan. You do not host a Python proxy.

Auth is a Cloudflare API token on Authenticated Gateway. BYOK stores provider keys in Secrets Store so the app stops shipping OpenAI secrets. Persistent logs sit on the gateway. Logpush exports them encrypted. Access can put a user id on the request as cf.user_id.

Key features:

• Authenticated Gateway with a Cloudflare API token
• BYOK provider keys in Secrets Store
• Persistent logs, plus Logpush on Workers Paid
• Caching, rate limiting, retries, fallback, DLP, and Guardrails on the public docs

Why we like it:

If the estate is already Cloudflare, this is the LLM front door that does not invent a new control plane. Logs and keys stay in the same account you already open.

Limits:

Docs say an AI Gateway Run token is account-scoped. It can hit every gateway in the account, including BYOK keys. Log storage is 100,000 logs on Workers Free and 10 million per gateway on Workers Paid. You cannot self-host the data plane.

License or pricing: Commercial. Core features free on all plans. Unified Billing adds a 5% fee on purchased credits.

Portkey is the managed control plane. Universal API, virtual keys, observability, guardrails, prompt management, budgets. Palo Alto now sells this as Prisma AIRS AI Gateway. The product people still search is Portkey.

The gateway core is MIT. You can run it with npx @portkey-ai/gateway. Hybrid puts the data plane in your network and leaves the control plane hosted. That is the production pattern they sell.

Key features:

• Virtual keys, RBAC, and budget limits on the public docs
• Logs, traces, and PII redaction on the request path
• Fallbacks, load balancing, retries, simple and semantic cache
• MIT gateway plus hosted and hybrid enterprise

Why we like it:

When a GenAI team has to issue keys to the rest of the company, a control plane with virtual keys and a log you can filter is the job. The OSS gateway is there if you will run it.

Limits:

Developer is 10k recorded logs a month and 3-day log retention. Production is $49/month for 100k logs. The features that look like a secrets program (SSO, VPC, custom retention) sit on Enterprise. The OSS gateway is routing. The dashboard is the product.

License or pricing: MIT gateway. SaaS Developer free, Production $49/month, Enterprise quoted.

LiteLLM is the OSS proxy most teams actually run. OpenAI-compatible. 100+ providers. Virtual keys, spend tracking, fallbacks, and a master key that mints the rest. Virtual keys need Postgres. The enterprise/ directory is a separate license.

GHSA-r75f-5x8p-qvmc, CVE-2026-42208, is the reminder. Affected versions mixed the caller-supplied key into the SQL text. Patch is 1.83.7. Rotate every credential the proxy stored. March 2026 also burned 1.82.7 and 1.82.8 on PyPI. The package is a secrets store. Treat it like one.

Key features:

• Virtual keys with model allow-lists, budgets, and spend tracking
• Fallbacks, load balancing, and an OpenAI-compatible endpoint
• You host the proxy, the database, and the log callbacks
• SSO, SCIM, and audit logs on the paid enterprise path on the public docs

Why we like it:

When you will operate the box and you want virtual keys without a SaaS in the data path, this is the default OSS name. Pin the version. Watch the advisory feed.

Limits:

You staff Postgres, the master key, and upgrades. Two 2026 incidents hit the same project. Enterprise identity is paid. A key with no user_id inherits nothing from the admin who minted it. Read their virtual-key docs before you assume RBAC.

License or pricing: MIT outside enterprise/. You run it. Enterprise quoted.

Helicone started as the log. Swap the base URL, add a header, see cost, latency, and the prompt. The later AI Gateway is a Rust router with caching, rate limits, fallbacks, and that same dashboard. Cloud or Docker.

The observability repo is Apache-2.0. The ai-gateway LICENSE file is GNU GPL v3, even though the README badge still says Apache. Mintlify acquired Helicone on 3 March 2026. First-party post: services stay live in maintenance mode, with security updates and new models still shipping. That is a roadmap risk you name in the buy.

Key features:

• One-line proxy logging of prompts, completions, tokens, and cost
• Cloud or self-hosted AI Gateway with routing and cache
• Sessions, users, custom properties, and HQL on the public docs
• Hobby 10k requests free; Pro from $79/month on the public list

Why we like it:

When the hole is “we cannot see the spend,” this is the fastest log. Policy can wait. The gateway is there if you want routing on the same box.

Limits:

Maintenance mode after the Mintlify deal. Self-host docs have been a community complaint. Confirm the gateway license with the LICENSE file. The README badge still says Apache. It is a log first. Hard budgets and virtual-key programs are stronger on LiteLLM and Portkey.

License or pricing: Apache-2.0 (helicone/helicone). GPLv3 (ai-gateway LICENSE). Cloud Hobby free, Pro $79/month, Team $799/month, Enterprise quoted.

Kong AI Gateway is the AI plugin slice on Kong Gateway. AI Proxy translates OpenAI-shaped requests onto OpenAI, Azure, Bedrock, Anthropic, Gemini, and more. Later versions add MCP and agent-to-agent traffic on the same runtime. The sibling API gateways list is the classic TLS and JWT job. This entry is the LLM plugins.

OSS covers AI Proxy and a handful of prompt plugins. Token quotas, semantic cache, PII sanitizer, semantic prompt guard, and AI Proxy Advanced (multi-model load balancing) sit on Enterprise or a paid Konnect tier. You still pay the model provider.

Key features:

• AI Proxy on a Kong route you already know how to operate
• Token quotas, PII sanitization, and prompt guards on paid plugins
• LLM, MCP, and A2A on one control plane on the public docs
• Self-host the data plane or run Konnect

Why we like it:

If Kong already terminates the rest of the APIs, adding an AI plugin is cheaper than a second proxy. Policy language stays the one the platform team already reviews.

Limits:

The interesting AI controls are paid. A team that does not run Kong should not buy Kong to get a LiteLLM substitute. MCP and A2A are extra surface. Keep that inventory on AI Security.

License or pricing: Apache-2.0 core and basic AI plugins. Advanced AI plugins Enterprise / Konnect. Quoted.

TrueFoundry is the lesser-known pick because it has a live AI Gateway product page, public tiers, and a VPC / on-prem / air-gap story. OpenRouter is a model marketplace. Databricks Mosaic AI Gateway is real, and it is a Databricks-shaped door. TrueFoundry is the standalone gateway we kept.

One endpoint, virtual keys, RBAC, quotas, guardrails, request logs, routing, and fallbacks. SaaS, hybrid, or you host the gateway plane. The product is proprietary. Self-hostable is not the same as open source.

Key features:

• Unified API across 1,600+ models they claim, including self-hosted vLLM
• RBAC, SSO, per-team keys, token and dollar quotas
• Input and output guardrails, PII filtering, full request logs
• SaaS, gateway-plane-only, or control plane plus gateway in your VPC

Why we like it:

When procurement wants a named AI gateway in the VPC and will not run LiteLLM as the production secrets store, this is the commercial plane with a first-party product URL.

Limits:

No public source tree to audit. Self-host still talks to their auth and licensing for emails and request counts, on the docs we read. MCP and agent features are in the pitch. This page still scores it as the LLM proxy.

License or pricing: Commercial. Public list includes a free 50k-request starter and Pro from $499/month. Self-host is Enterprise. Infra they estimate around $600 to $1,000/month on top.