LlamaForge plugin
LlamaForge plugin and MCP server: an agent checks which model fits your GPU, loads it into llama.cpp and hands it routine work
- 124 ★ on the repository
- Last commit
- MIT
Broad access
Why this level
- The `pi_run` tool in `full` mode runs shell commands in your project; by default it only reads files.
- The agent loads and unloads models, downloads large files and changes model settings.
- The panel installer runs through `curl | sh` or `irm | iex` without admin rights.
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add plugins/llamaforgeDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- The `pi_run` tool in `full` mode runs shell commands in your project; by default it only reads files.
- The agent loads and unloads models, downloads large files and changes model settings.
- The panel installer runs through `curl | sh` or `irm | iex` without admin rights.
This is third-party code. Review the repository files before installing.
What it does
LlamaForge is a browser panel that installs the official llama.cpp and drives its `llama-server` router: a model list, downloads from Hugging Face, a fit estimate against GPU memory, flag editing. For agents it ships a `llamaforge` skill and a stdio MCP server in `backend/mcp_server.py`. The skill explains how to check that the panel is running, how to connect the tools and what to do in common tasks: find out what fits the GPU, download a model, speed up startup with Auto-Forge, hand routine work to a local model. The MCP server exposes 17 tools: `status`, `list_models`, `load_model`, `unload_model`, `fit_check`, `search_models`, `download_model`, `ask`, `pi_run`, `forge_start` and more. A compatibility layer translates Anthropic Messages API requests into the llama.cpp format, so Claude Code can run on a local model.
Who it is for. GPU owners who run local models through llama.cpp and want to manage them from Claude Code or Codex.
Good fit when
- An agent has to find out which quant of a Hugging Face model fits into GPU memory.
- Routine work such as log triage goes to a local model to save tokens.
- Claude Code or Codex is pointed at a local model with one click in the panel.
- A model failed to load and you need a diagnosis with the tail of the router log.
- You want faster generation by measuring layer placements and speculative drafters with Auto-Forge.
Not a fit when
- Models run only through cloud APIs.
- You need a mature desktop model manager: LlamaForge is an early preview.
- The panel cannot stay running on the machine: without it the MCP tools do not respond.
Example request
Find out which Qwen quant fits my GPUs with room left for context and show the size in gigabytes.Limitations
The project is an early preview: Windows with NVIDIA is the best tested path, Linux and macOS pass CI but saw little real hardware. The panel has to be running or the MCP tools cannot connect. The `pi_run` tool needs the pi agent installed separately. Auto-Forge unloads every model for the run and `forge_apply` changes model settings. The installers run through `irm ... | iex` or `curl ... | sh`, so read the script before running it.
How to disable. Remove the MCP entry with `claude mcp remove llamaforge`, the skill from the agent skills folder and the `env` block from `~/.claude/settings.json` if you pointed Claude Code at a local model. Stop the panel and delete its folder separately.
FAQ
How do I connect the LlamaForge MCP server to Claude Code?
Open Setup in the panel, then MCP server, and copy the ready `claude mcp add --scope user llamaforge -- <python> <script>` command. The panel has to run while the agent uses the tools.
How do I install the skill?
Run `npx skills add dadwritestech/LlamaForge` or install the Claude Code plugin. If the panel is missing, the skill describes it and asks before installing.
Can Claude Code run on a local model?
Yes. Agent setup writes an `env` block with `ANTHROPIC_BASE_URL` pointing at the local panel address into `~/.claude/settings.json`. For Codex and pi.dev the panel prepares its own configs.
What does `pi_run` do?
It hands a task to the pi agent running on a loaded local model and returns its final answer. By default pi only reads files, `edit` allows editing, `full` allows shell commands.
What is Auto-Forge?
A measuring mode: it tries layer placements, KV cache types and speculative drafters, checks that a drafter yields the same tokens and, on your command, writes the best setup to the model settings.
Discussion
No questions or reviews yet.
Related
Pluginsllmman runs Claude Code, Codex and OpenCode on a local or hosted model with one command and stores models as OCI images
lilbee runs local models across several GPUs and gives agents MCP search over your files with a file and line citation
A local model server for Apple Silicon Macs in OpenAI, Anthropic and Ollama formats: Claude Code, Codex and OpenCode connect with one command
LM Studio commands list downloaded and loaded models, control the local API server and stream application logs in a terminal.
A CLI and Python library for OpenAI, Claude, Gemini, local and other models: prompts, chats, tools, embeddings and a SQLite log
LM Studio is a desktop app for running open models locally, with a chat, a local server in OpenAI and Anthropic formats, and MCP support.
README in short
The panel runs in a browser and runs no models itself: llama.cpp does the inference. If models were downloaded through Ollama, the panel finds them and runs the same files without copying.