LiteLLM

LiteLLM gives you one OpenAI-style API in front of 100+ model providers: OpenAI, Anthropic, Gemini, Bedrock, Azure, Ollama and the rest. Use it as a Python SDK, or run the proxy as a gateway with virtual keys, per-team spend tracking, budgets, rate limits, fallbacks and load balancing. The core is MIT; an enterprise folder is proprietary.

Self-host the proxy with Docker and Postgres. It moves fast: a new minor version roughly every week with only the four most recent supported, so a pinned version falls out of support in about a month. A Rust rewrite is underway, targeting the end of 2026. Overhead is small for normal requests (about 8 ms at p95 in their own benchmark), but the Python proxy struggles with very large prompts under load.

Everything most teams need is free, and SSO is free for up to 5 users. Enterprise adds SSO beyond that, SCIM, audit logs, secret managers, IP allowlists, team-scoped guardrails and 24/7 SLAs, priced by annual request volume with no public numbers. Small and mid-size teams: free. Large orgs with compliance needs: talk to sales.

The catch: it holds every API key you own, and its security record demands attention. Two poisoned releases (1.82.7 and 1.82.8) were live on PyPI for about 40 minutes in March 2026, and critical advisories kept coming through September. Use the official Docker image, pin it, and patch fast.