liteLLM

Overview

LiteLLM is an open‑source gateway that unifies access to 100+ LLMs through a single OpenAI‑compatible API, enabling provider fallback, cost tracking, tag‑based budgeting, guardrails, observability, and on‑prem or cloud deployment with a lightweight SDK.

From the official site

Put your full AI stack behind one key. Track and cap LLM spend, route to the right model, and self-host anywhere — even air-gapped. 140+ providers, 1,892 models, 53K+ GitHub stars.

The text above is quoted from this tool’s official website — the vendor’s own words.

Official FAQ

What is LiteLLM?

LiteLLM is an open-source AI gateway (LLM proxy) that puts your entire AI stack behind one OpenAI-compatible API. It tracks and caps spend per key, team, and model, routes each request to the right model, and can be self-hosted anywhere, including fully air-gapped.

Is LiteLLM free and open source?

Yes. The LiteLLM gateway is MIT-licensed, free to self-host, and used in production by 53,000+ GitHub stargazers' teams. An Enterprise tier adds SSO, RBAC, audit logs, SCIM, and dedicated support on top of the same open-source core — not a different product.

What is an AI gateway (LLM gateway)?

An AI gateway, also called an LLM gateway or LLM proxy, is a single control point in front of many LLM providers. It gives applications one API to call, and gives platform teams centralized cost tracking, budgets, routing, caching, and governance across every model.

Can I self-host LiteLLM in an air-gapped environment?

Yes. LiteLLM self-hosts in your own cloud or a fully air-gapped environment with no telemetry, so data never leaves your infrastructure. Deploy with an official Helm chart or Terraform module, on your own Postgres and Redis, scaling with Kubernetes autoscaling.

How many LLM providers and models does LiteLLM support?

LiteLLM supports 140+ LLM providers and 1,892 models through one OpenAI-compatible interface, so you can add or swap models without changing application code.

How does LiteLLM control LLM costs?

LiteLLM attributes every request automatically and gives each team a virtual key with a hard budget. When a key hits its cap, requests stop, so a runaway job or leaked key cannot quietly run up the bill. Spend can be exported to Prometheus, Datadog, or CloudZero for chargeback.

These questions and answers come from the tool’s own structured data, not written by us.

How to use it

1. Open litellm.ai, register and review the available capabilities and quotas
2. Create a project or obtain access credentials from the documentation
3. Complete a minimal call in your local or test environment using the examples
4. Once validated, integrate into the real project and expand usage gradually