GPT-Load CLI
A self-hosted gateway for multiple LLM keys and providers: rotation, failover, request logs and usage accounting
- 7,213 ★ on the repository
- Last commit
- MIT
Broad access
Why this level
- The service stores and decrypts LLM provider API keys and credentials
- It requires running your own server with network access
Install
- Works in
Manual install
git clone --depth 1 https://github.com/tbphp/gpt-load.git
cd gpt-load
cp .env.example .env
docker compose up -dThe primary method, requires Docker and Docker Compose.
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- The service stores and decrypts LLM provider API keys and credentials
- It requires running your own server with network access
This is third-party code. Review the repository files before installing.
What it does
GPT-Load is a Go server that sits between your apps or agents and LLM providers: OpenAI, Anthropic, Gemini and others over an OpenAI-compatible protocol. It pools several API keys and subscription accounts behind one entry point, distributes requests across them, fails over to another key on an error or rate limit, and logs requests with a cost estimate. Configuration happens through a web console: add a channel with keys, create a model group, and issue a client access key with its own limits. Data is stored in SQLite by default, or in MySQL or PostgreSQL.
Who it is for. For anyone juggling several LLM provider keys and accounts for a team or their coding agents who wants one entry point instead of switching manually.
Good fit when
- You need to spread load across several keys or accounts of the same provider
- You need automatic failover to a backup key on an error or rate limit
- You need one request log and a cost estimate across all keys and models
Not a fit when
- You use a single key with a single provider and need no failover
- You do not want to run and maintain your own server
Example request
Deploy a local GPT-Load gateway and connect two Anthropic keys to it with automatic failover on a rate limitLimitations
Requires Docker and Docker Compose, or a prebuilt binary for your platform. The encryption.key file must be kept together with the database: if it is lost, stored channel credentials cannot be recovered, and this version does not support master key rotation.
How to disable. Stop the service with docker compose stop and remove the containers and the gpt-load-data volume if needed, or terminate the native binary process.
FAQ
Can it run without Docker?
Yes, the Releases page has portable binaries for Linux, macOS and Windows, and a Windows installer with a service is also available.
Is the service exposed externally by default?
No, by default it listens only on the loopback address 127.0.0.1; external access requires explicitly setting HOST=0.0.0.0.
Discussion
No questions or reviews yet.
Related
CLI toolsA self-hosted gateway between AI apps and models, MCP tools and A2A agents: policy, budgets, routing and an audit trail
The GoModel gateway puts MCP servers behind one protected address and keeps their tokens out of the agent's reach.
Agentgateway
agentgateway
A proxy for agentic traffic: routes and secures connections to MCP servers, LLMs, and other agents over A2A
A self-hosted gateway routes internal MCP tools, authenticates users and records requests for later review.
The CLI starts a language-model proxy and a request tracking service for debugging applications across model providers.
setup-gateway writes an OpenAI-compatible gateway into the Claude Code, OpenCode and Codex CLI configs in one pass, with a backup.
README in short
The README describes GPT-Load as a self-hosted AI gateway for multi-channel, multi-credential setups: pooling API keys and subscription accounts, traffic scheduling, failover, request logs and usage accounting in a web console. Deployment goes through Docker Compose with SQLite by default, or MySQL and PostgreSQL through one DATABASE_DSN connection string. Native binaries and a Windows installer are documented separately. MIT licensed.