SkillFoxx

Shimmy CLI

A local GGUF model server accepts OpenAI-style requests so applications can be tested on hardware you control.

  • 6,014 ★ on the repository
  • Last commit
  • Apache-2.0
CLI

Extended access

Why this level

  • The server reads model files, consumes GPU resources and processes submitted text.
  • A network bind broadens the clients able to send requests to your hardware.
All reasons and checks

michael-a-kuykendall/shimmyAuthor:Michael-A-Kuykendall

Install

In your terminal, with SkillFoxx CLI

npx skillfoxx add cli/shimmy

Detects the agents on your machine, shows the access level and pins the version.

Other ways to install

Access and checks

How we review

Extended accessThe tool runs code or makes network calls and only reads external data.

  • The server reads model files, consumes GPU resources and processes submitted text.
  • A network bind broadens the clients able to send requests to your hardware.

This is third-party code. Review the repository files before installing.

What it does

Shimmy loads a GGUF model file and exposes a text-generation HTTP API. Applications send a conversation to the local server and receive a complete or streamed response. The supported surface includes chat, text completion and model listing. Airframe runs computation through WebGPU, a cross-platform GPU interface. GGUF metadata supplies model configuration. Documentation lists tested model and weight-quantization combinations; recognizing an architecture does not by itself establish full compatibility.

Who it is for. Application developers evaluating local model inference through the familiar OpenAI request format.

Good fit when

  • An application already uses OpenAI API requests, and its developer changes the endpoint to test on a local GPU.
  • A researcher has downloaded a verified GGUF model and runs it locally.
  • A team processes test texts on its own hardware without sending generation requests to a cloud service.
  • A chat-interface author tests displaying an answer as tokens arrive.

Not a fit when

  • You need every feature of the hosted OpenAI API.
  • Your required model and quantization combination is absent from the tested list.
  • The deployment depends on the former Shimmy 1.x llama.cpp or MLX backends.

Example request

Serve the selected tested GGUF model on loopback only and send a short test conversation through Shimmy.

Limitations

Model files and sufficient device memory are separate requirements; context length and quantization affect memory usage. OpenAI compatibility covers the documented API endpoints, so compare a client’s requirements with that list. SafeTensors loading exists, but full Airframe-native execution remains planned work. Network binds require `SHIMMY_API_KEY`, and browser clients must also be listed in `SHIMMY_ALLOWED_ORIGINS`.

How to disable. Stop the Shimmy process and remove its address from the application settings. Downloaded model files remain on disk.

FAQ

How do I install Shimmy?

Run `cargo install shimmy --locked`. Model acquisition, GPU setup and memory sizing are covered in the [quickstart](https://github.com/Michael-A-Kuykendall/shimmy/blob/main/docs/quickstart.md).

How do I serve my model file?

Run `shimmy serve --model-path /absolute/path/to/model.gguf --bind 127.0.0.1:11435` with the actual file path. Loopback keeps access on this machine.

How do I list models?

Run `shimmy list --short` and match the requested model name to its output.

Will every GGUF model work?

Do not assume so. Choose a tested model and quantization pair from [Supported Models](https://github.com/Michael-A-Kuykendall/shimmy/blob/main/docs/SUPPORTED_MODELS.md); architecture recognition alone is insufficient.

Where should a chat request go?

Quick Start uses `http://127.0.0.1:11435/v1/chat/completions`. Keep it local for the test and supply the name of a model you actually loaded.

Discussion

No questions or reviews yet.

CLI tools
Official

A local model server for Apple Silicon Macs in OpenAI, Anthropic and Ollama formats: Claude Code, Codex and OpenCode connect with one command

CLIBroad access2.4K

Local coding agent with a terminal UI: it plans, drafts several candidates, tests them in a sandbox and repairs, so compact open models can handle code

CLIBroad access2.1K

lilbee runs local models across several GPUs and gives agents MCP search over your files with a file and line citation

MCP serverExtended access70

A skill for Claude Code and Gemini CLI that runs LLM fine-tuning from dataset to GGUF export, on NVIDIA GPUs and Apple Silicon

SkillExtended access283
Russian stack

A local proxy exposing GigaChat through OpenAI- and Anthropic-compatible APIs so it can drive Claude Code, aider and other agents

CLIExtended access

A CLI and Python library for OpenAI, Claude, Gemini, local and other models: prompts, chats, tools, embeddings and a SQLite log

CLIExtended access12.8K

README in short

The 2.x line moved to Airframe and removed older backends. Separate guides cover migration, extended context and GPU troubleshooting.

README badge

Report a problem