New API: Unified LLM Gateway & Billing

1. Project Overview

New API is an open-source, next-generation LLM gateway and AI asset management system that lets organizations aggregate multiple AI model providers behind a single unified API, while adding enterprise-grade authentication, usage analytics, and cost accounting on top.

2. Background & Positioning

New API was built on top of the well-known One API project, extending it with a redesigned interface, richer billing capabilities, and features tailored for organization-level, self-hosted deployments. The core mission is to give teams and businesses a single, controllable entry point for calling many different large language model (LLM) providers — instead of juggling separate SDKs, API keys, and billing dashboards for each vendor.

Compared with similar LLM proxy/gateway projects, New API differentiates itself by combining API-format translation (OpenAI, Claude, Gemini, and more) with a full-featured admin console: per-user quotas, multi-channel load balancing, cache-aware billing, and support for community login providers (Discord, LinuxDO, Telegram, OIDC) that are popular in self-hosted and community-run deployments.

3. Feature Categories

🖥️ Admin Console & UI
Modern, multi-language web dashboard (Chinese, English, French, Japanese) for managing channels, users, and billing. Includes visual usage analytics, statistical reporting, and a redesigned UI compared to its One API predecessor. Purpose: give administrators full visibility and control without touching the database directly.

💳 Billing & Monetization
Internal top-up wallet system with EPay and Stripe integration, request-level usage-based cost accounting, and cache billing support for providers like OpenAI, Azure, Claude, DeepSeek, and Qwen. Purpose: let operators resell or internally charge for LLM access with accurate, auditable costs.

🔌 API Format Compatibility
Supports OpenAI Chat/Responses format, Claude Messages API, Google Gemini format, and rerank APIs (Cohere, Jina), with automatic format conversion between incompatible upstream and downstream formats. Purpose: let client applications keep using their preferred SDK while the gateway talks to whichever provider is actually configured.

🔀 Routing & Reliability
Weighted random channel distribution, automatic failover/retry on upstream failure, and per-user model rate limiting. Purpose: keep service available and fair even when individual upstream providers degrade or rate-limit requests.

🔑 Authentication & Access Control
Multiple login methods (Discord, LinuxDO, Telegram, OIDC) plus token- and user-level permission controls. Purpose: fit both community/open self-hosted deployments and controlled internal-organization deployments.

4. Key Highlights

Multi-provider aggregation: One gateway endpoint fronts OpenAI, Claude, Gemini, Azure, Midjourney-Proxy, Suno, and other custom upstreams, covering chat, image, audio, embeddings, and video interfaces.
Format translation layer: Automatically converts between OpenAI, Claude, and Gemini request/response formats so client code doesn't need to be rewritten per provider.
Cache-aware cost accounting: Understands prompt-caching billing behavior for major providers, so cost reports reflect actual (discounted) spend rather than sticker price.
Reasoning effort configuration: Exposes reasoning-effort controls across multiple model families that support them, useful for cost/latency tuning on reasoning-capable models.
Enterprise-friendly billing: Built-in Stripe and EPay top-up flows plus flexible billing policies, so the gateway can act as a self-service, pay-as-you-go internal platform.
Resilient routing: Weighted channel distribution with automatic retry means a single failing upstream key or provider doesn't take down the whole gateway.

5. Use Cases by Role

General developers: Point existing OpenAI/Claude/Gemini-compatible SDKs at a single New API endpoint and swap or mix backend providers without changing application code.

DevOps/SRE: Deploy New API as a self-hosted gateway (Docker/Docker Compose) in front of multiple LLM vendors to centralize secrets, monitor usage, and add failover so no single provider outage causes an app-wide incident.

Project managers: Use the built-in usage dashboards and cost accounting to track spend per team, project, or API key, and set quotas to keep AI usage within budget.

6. Getting Started

Find what you need — start with the project README and Wiki for feature explanations and configuration references:

https://github.com/QuantumNous/new-api

Install/integrate — the fastest path is Docker:

docker run --name new-api -p 3000:3000 -v ./data:/data calciumion/new-api:latest

Docker Compose is recommended for production, using SQLite for simple local setups or MySQL 5.7.8+/PostgreSQL 9.6+ for shared/remote database deployments.

Contribute — fork the repository, create a feature branch, and open a pull request:

git clone https://github.com/QuantumNous/new-api.git

7. Project Structure

new-api/
├── controller/     # HTTP handlers for admin API and gateway endpoints
├── relay/          # Core request relay/proxy logic to upstream model providers
├── model/          # Database models (users, channels, tokens, logs)
├── middleware/      # Auth, rate limiting, and request middleware
├── web/            # Frontend admin console (multi-language UI)
├── docker-compose.yml
└── README.md

The relay/ directory is the heart of the project — it contains the format-translation and provider-dispatch logic that lets a single request be routed to and reshaped for different upstream LLM APIs.

8. Related Ecosystem

New API builds on the architecture pioneered by One API, and integrates with upstream model providers including OpenAI, Anthropic (Claude), Google (Gemini), Azure OpenAI, Midjourney-Proxy, and Suno API. It also supports login via community platforms such as Discord, LinuxDO, and Telegram, and payment integration via Stripe and EPay, making it a hub that connects LLM vendors, identity providers, and payment processors.

9. License

✅ Free to use, modify, and self-host under the AGPLv3 license, including for commercial and internal enterprise deployments.
✅ You may redistribute modified versions.
❌ You may not remove attribution or the visible link back to the original repository in modified/redistributed versions.
ℹ️ AGPLv3 requires that if you run a modified version as a network service, you must make the modified source code available to users of that service — review the license text before deploying a customized fork commercially.

10. FAQ

Q: Does New API require a specific database?
A: No — it supports SQLite for simple local deployments, and MySQL 5.7.8+ or PostgreSQL 9.6+ for production/multi-node deployments.

Q: Can I run New API across multiple nodes?
A: Yes, but all nodes must share the same database and an identical SESSION_SECRET environment variable, and Redis is recommended as a shared cache layer.

Q: Which model providers are supported?
A: OpenAI, Claude, Google Gemini, Azure, Midjourney-Proxy, Suno, and other custom-configured upstream channels, covering chat, image, audio, embedding, and video interfaces.

Q: Is New API free to self-host?
A: Yes, it is open source under AGPLv3 and free to deploy; billing/top-up features let operators charge their own end users if desired.

11. Quick Links

Repository: https://github.com/QuantumNous/new-api
Docker image: calciumion/new-api
Contributing: open a pull request against the main repository
Community/discussions: GitHub Issues and Discussions on the repository page

12. Summary

New API is a mature, actively maintained self-hosted gateway for consolidating access to multiple LLM providers behind one API, with production-grade billing, routing, and authentication features. It is best suited for teams, communities, or businesses that need to manage multi-provider AI costs and access centrally rather than integrating each provider's API separately.