MTPLX CLI
A local model server for Apple Silicon Macs: Qwen 3.8 and Gemma 4 with MTP decoding and OpenAI and Anthropic APIs for Claude Code, Codex and Cursor
- 2,625 ★ on the repository
- Last commit
- Apache-2.0
Broad access
Why this level
- Sets up its own Python environment in `~/.mtplx` and downloads models up to 170 GB from Hugging Face
- Brings up a local server on `127.0.0.1:8000` that takes agent requests to the model and keeps a session cache on disk
- Fan control (`mtplx max --install`) asks for `sudo`, and Forge publishes models to the Hub with your Hugging Face token
Install
- Works in
Manual install
brew install youssofal/mtplx/mtplxNeeds an Apple Silicon Mac on macOS 14 or newer. Start model selection with mtplx start. The app with its installer is downloaded from mtplx.com.
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- Sets up its own Python environment in `~/.mtplx` and downloads models up to 170 GB from Hugging Face
- Brings up a local server on `127.0.0.1:8000` that takes agent requests to the model and keeps a session cache on disk
- Fan control (`mtplx max --install`) asks for `sudo`, and Forge publishes models to the Hub with your Hugging Face token
This is third-party code. Review the repository files before installing.
What it does
MTPLX is a local model server for Apple Silicon Macs: the app and the `mtplx` command line run Qwen 3.8 Flash Next, Qwen 3.8 27B, Ternary Bonsai 2 27B, Qwen 3.6, Qwen 3.5 and Gemma 4 on your computer. It gets its speed from the model's own multi-token prediction (MTP) heads: the model drafts several tokens ahead, one pass verifies the draft, and tokens are committed by exact rejection sampling with a residual correction. The author measured a 1.6x speedup on a 16 GB M4 Mac mini and 2.24x on an M5 Max MacBook Pro. `mtplx start` offers a choice of model, mode and surface. `mtplx serve` brings up a server on `127.0.0.1:8000` with the OpenAI routes `/v1/chat/completions`, `/v1/responses`, `/v1/completions` and `/v1/models`, and the Anthropic `/v1/messages` with streaming and tool calls. Claude Code, Codex, Cline, Cursor, OpenCode and any client that speaks these APIs connect to it. When needed, the same process serves `/v1/embeddings` and `/v1/rerank` for search and agent memory. The app picks a model that fits your memory, downloads it from the Youssofal catalog on Hugging Face, sets up its own Python environment and runs `mtplx tune`: the command measures plain generation against MTP depths 1 to 3 on your Mac and saves the fastest.
Who it is for. For developers on Apple Silicon Macs who want to run coding agents on a local model with the code staying on the machine and the best speed.
Good fit when
- Code must not go to external APIs, and a Claude Code, Codex or OpenCode agent has to run on a local model
- A Mac with 32 GB of memory or more and you want Qwen 3.8 27B Optimized Speed, which the author recommends for coding
- A 16 GB Mac: the app offers Ternary Bonsai 2 27B, which takes 8.85 GB and runs with an 8,192-token window
- You want one local server for chat and for embeddings with reranking, so agent memory needs no second server
- You want to measure whether MTP speeds up your own Mac: `mtplx tune --retune` compares it with plain generation
Not a fit when
- The computer runs Linux, Windows or an Intel processor: the project targets Apple Silicon and macOS 14 or newer, and the author points Linux users to vLLM
- You need a model without MTP heads: some models run in plain generation mode and get no speedup
- You have under 16 GB of memory and need a large agent context: the author notes agent clients such as OpenCode and Hermes need 18 GB or more
- You want a catalog of ready models with a download manager and a chat: Ollama and LM Studio fit that
- The server must be reachable from other machines on the network: the documentation describes a launch on `127.0.0.1:8000`
Example request
Start a local MTPLX server on my Mac with the Qwen 3.8 27B model and point Claude Code in this project at it, so the code stays off the cloudLimitations
- Models are large: Flash Next at 4-bit takes 106–115 GB on disk and 74–83 GB of memory, the Optimized Quality build weighs about 170 GB and, according to the author, has not yet run on a 256 GB Mac. Qwen 3.8 27B Optimized Speed takes 20.4 GB.
- The author recorded the speeds in the table on an M5 Max MacBook Pro with 128 GB and fans at maximum. Other Macs give other results, which is why the project ships the `mtplx tune` command.
- The app on first launch and the `mtplx max --install` command set up fan control through ThermalForge with one `sudo` prompt. The fan-backed mode is optional.
- The Responses route for Codex does not support hosted tools: the server answers a `web_search` request with a 400 error until the tool is turned off in the client.
- The license is Apache-2.0, but a product or service built on MTPLX must show "Powered by MTPLX" with a link inside the product or its docs; a mention on a website is not enough. Model weights follow the licenses of the original model authors.
How to disable. Stop the server with `mtplx stop`. `mtplx remove <hf-repo>` deletes a downloaded model. If you installed through Homebrew run `brew uninstall mtplx`, and for a pip install use `python3 -m pip uninstall mtplx`. The script installer puts its environment in `~/.mtplx/venv` and a launcher in `~/.local/bin/mtplx`, and you can delete those paths by hand. The app is removed like any other application.
FAQ
How do I run Qwen 3.8 27B locally on a Mac?
Install `mtplx` and run `mtplx serve --model Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed`. The 4-bit build takes 20.4 GB and fits a Mac with 32 GB of memory or more. For M1 and M2 the app picks the FP16 build with 16-bit tensors by itself.
How do I connect Claude Code to a local model on a Mac through MTPLX?
Run `mtplx serve` and set the address `http://127.0.0.1:8000` in the client. The Anthropic-compatible `/v1/messages` route supports streaming and tool calls in both styles. Codex uses `/v1/responses`, but a request with `web_search` gets a 400 error.
Which model fits a Mac with 16 GB of memory?
The app recommends Ternary Bonsai 2 27B: the pack weighs 8.85 GB and on 16 GB gets an 8,192-token window and a memory peak of 11.8 GiB. Agent clients such as OpenCode and Hermes need 18 GB or more. The 9B Qwen 3.5 and MiMo V2.6 packs also fit.
Does MTP decoding change the model's answers?
According to the author, no. Tokens are accepted by exact rejection sampling, so the distribution matches plain decoding at any temperature. On version 2.11.3 the author compared a thousand four-token samples from the fast and plain paths, and the difference stayed within the plain path's own noise.
What does the mtplx tune command do?
It runs the real model on your Mac at MTP depths 1 to 3 and compares with plain generation. If a depth wins it is saved, and if none wins nothing is saved. For a 9B model on a 16 GB M4 Mac mini the speed rose from 14.4 to 23.0 tokens per second.
Does MTPLX run on Linux or Windows?
No. The project is built on MLX for Apple Silicon, and Intel processors and systems other than macOS 14 or newer are not claimed. For Linux the author points to vLLM.
Discussion
No questions or reviews yet.
Related
CLI toolsA local model server for Apple Silicon Macs in OpenAI, Anthropic and Ollama formats: Claude Code, Codex and OpenCode connect with one command
The Apple Intelligence model built into a Mac as a terminal command and a local OpenAI-compatible server: no keys, no cloud
Local memory for Claude Code, Codex and Cursor Agent: learns from your activity on a Mac and gives agents context with source receipts over MCP
LM Studio is a desktop app for running open models locally, with a chat, a local server in OpenAI and Anthropic formats, and MCP support.
An MCP server that lets Claude Code, Codex and other agents call Gemini, GPT, Grok and local models for review, debugging and consensus
Local coding agent with a terminal UI: it plans, drafts several candidates, tests them in a sandbox and repairs, so compact open models can handle code
Comparison with similar tools
Entries that share the main search query, and how they differ.
| Entry | Official | Repository stars | Main difference |
|---|---|---|---|
| MTPLX (this entry) | yes | 2,625 | |
| mlx-serve | yes | 2,030 | a local server for Mac in OpenAI, Anthropic and Ollama formats that attaches Claude Code, Codex and OpenCode with one command |
| Ollama | yes | 184,225 | runs open models on your computer and serves them to agents through a local OpenAI and Anthropic API |
| LM Studio | yes | an app with a chat and a local server in OpenAI and Anthropic formats and MCP support |
README in short
The repository front page opens with a table of measured speeds per model, then explains how to get the app and lists models with sizes and modes. Next come sections on the dashboard app, auto-tuning, Forge for building your own MTP models, the server with embeddings and reranking, a command quick reference and the Turbo, Sustained and Burst modes. The repository also holds release notes, a measurement history, a troubleshooting guide and the macOS app written in Swift.