GPT-Load
A self-hosted gateway for multiple LLM keys and providers: rotation, failover, request logs and usage accounting
High risk
We rate an entry high when the tool writes to external systems, handles money, production databases or secrets, or runs arbitrary commands. The CLI installs it only with your consent.
Why this level
- The service stores and decrypts LLM provider API keys and credentials
- It requires running your own server with network access
Install
Manual install
git clone --depth 1 https://github.com/tbphp/gpt-load.git
cd gpt-load
cp .env.example .env
docker compose up -dThe primary method, requires Docker and Docker Compose.
This is third-party code. Review the repository files before installing.
What it does
GPT-Load is a Go server that sits between your apps or agents and LLM providers: OpenAI, Anthropic, Gemini and others over an OpenAI-compatible protocol. It pools several API keys and subscription accounts behind one entry point, distributes requests across them, fails over to another key on an error or rate limit, and logs requests with a cost estimate. Configuration happens through a web console: add a channel with keys, create a model group, and issue a client access key with its own limits. Data is stored in SQLite by default, or in MySQL or PostgreSQL.
Who it is for. For anyone juggling several LLM provider keys and accounts for a team or their coding agents who wants one entry point instead of switching manually.
Good fit when
- You need to spread load across several keys or accounts of the same provider
- You need automatic failover to a backup key on an error or rate limit
- You need one request log and a cost estimate across all keys and models
Not a fit when
- You use a single key with a single provider and need no failover
- You do not want to run and maintain your own server
Example request
Deploy a local GPT-Load gateway and connect two Anthropic keys to it with automatic failover on a rate limitLimitations
Requires Docker and Docker Compose, or a prebuilt binary for your platform. The encryption.key file must be kept together with the database: if it is lost, stored channel credentials cannot be recovered, and this version does not support master key rotation.
How to disable. Stop the service with docker compose stop and remove the containers and the gpt-load-data volume if needed, or terminate the native binary process.
Security check
- The service stores and decrypts LLM provider API keys and credentials
- It requires running your own server with network access
README in short
The README describes GPT-Load as a self-hosted AI gateway for multi-channel, multi-credential setups: pooling API keys and subscription accounts, traffic scheduling, failover, request logs and usage accounting in a web console. Deployment goes through Docker Compose with SQLite by default, or MySQL and PostgreSQL through one DATABASE_DSN connection string. Native binaries and a Windows installer are documented separately. MIT licensed.
FAQ
Can it run without Docker?
Yes, the Releases page has portable binaries for Linux, macOS and Windows, and a Windows installer with a service is also available.
Is the service exposed externally by default?
No, by default it listens only on the loopback address 127.0.0.1; external access requires explicitly setting HOST=0.0.0.0.
Related
An MCP server built into the Netdata agent: metrics, logs, alerts and live process, service and container data for an AI assistant
GitHub's official MCP server: code, issues, pull requests, Actions and security alerts straight from the agent
Agent Skills for Google products
Agent Skills for Google products and technologies
Official Google skill collection for working with Google Cloud, BigQuery, GKE, ads and analytics from an agent
AWS's official MCP server suite: docs, IaC, containers, serverless, databases, cost and monitoring