mlx-serve CLI
A local model server for Apple Silicon Macs in OpenAI, Anthropic and Ollama formats: Claude Code, Codex and OpenCode connect with one command
- 2,054 ★ on the repository
- Last commit
Broad access
Why this level
- The MLX Core app's agent mode runs shell commands on the Mac unless the sandbox with an isolated virtual machine is on
- The server opens port `11234`, and models can be shared across the local network through Bonjour
- Launchers write agent settings: Codex runs against your own home folder, and the fx entry lands in `~/.fx/settings.json`
Install
- Works in
Manual install
brew tap ddalcu/mlx-serve https://github.com/ddalcu/mlx-serveAdds the Homebrew tap. Then run `brew trust ddalcu/mlx-serve`.
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- The MLX Core app's agent mode runs shell commands on the Mac unless the sandbox with an isolated virtual machine is on
- The server opens port `11234`, and models can be shared across the local network through Bonjour
- Launchers write agent settings: Codex runs against your own home folder, and the fx entry lands in `~/.fx/settings.json`
This is third-party code. Review the repository files before installing.
What it does
mlx-serve runs language models on Apple Silicon Macs: MLX-format models and any GGUF from Hugging Face through an embedded llama.cpp. The server is written in Zig and needs no Python. On a single port, `11234`, it speaks four protocols: OpenAI (`chat/completions` and Responses), Anthropic Messages and Ollama. So Claude Code, Codex, OpenCode, Continue and Open WebUI connect without a proxy. `mlx-serve run gemma4` downloads a model, starts the server and opens a chat, `pull` only downloads, `list` shows what is on disk and `serve` serves every model by name. `mlx-serve launch claude` configures and starts an agent on a local model. It also supports `codex`, `opencode`, `pi`, `omp`, `hermes`, `aider`, `fx`, `grok` and `zcode`. The launcher reads the real context window from the server and writes the agent config into `~/.mlx-serve/<agent>/`, leaving your real configs alone. Codex and fx are exceptions: the first launches against your own Codex home, and for the second the launcher adds an entry to `~/.fx/settings.json`. Beyond text the server generates images, video, music, speech with voice cloning and 3D models. The MLX Core menu-bar app combines this with chat, an agent mode with ten built-in tools and an MCP catalog.
Who it is for. For Apple Silicon Mac owners who want to run open models locally and point coding agents at them without setting up a proxy or editing configs.
Good fit when
- You want Claude Code, Codex or OpenCode running on a local model: `mlx-serve launch <agent>` sets the agent up itself
- You have Ollama clients such as Raycast or Obsidian and want to point them at a faster engine without changing habits
- You need a local server speaking the OpenAI, Anthropic and Ollama formats on one port
- You want to generate images, music, speech or video on your own Mac through the same HTTP requests
- You want the server's web console: `http://localhost:11234` opens chat, generation and monitoring
Not a fit when
- You have an Intel Mac, macOS older than 26.2 and no NVIDIA card with driver 580 and CUDA 13: builds exist only for those systems
- Memory is short for the models you picked: LTX video asks for 24 GB and the 3D model for 16 GB
- You want Apple's built-in system model without downloading weights: `apfel` exposes it
- You just want to run a text model and nothing more: `ollama` is simpler in scope and runs on Windows, where mlx-serve does not
Example request
Install mlx-serve, download Gemma 4 and start Claude Code on that local model through mlx-serve launchLimitations
- It needs macOS 26.2 or newer on Apple Silicon. For Linux there are builds with NVIDIA CUDA 13, driver 580 or newer and cuDNN 9, and there is no Windows build.
- The server pins the context window at model load according to memory, and it is often well below the model maximum. `/v1/models` reports the real value, so avoid raising it by hand in an agent's settings.
- The `--api-key` key is not required from clients on the same computer: local connections pass without it.
- Speed figures and comparisons with other engines come from the author's measurements on an M4 Max. Models and tools for images, video, music and speech are downloaded separately, with sizes listed in the project table.
- MIT license, with bundled third-party parts keeping their own licenses, Apache-2.0 among them. The Mac App Store build cannot launch other apps and shows copy-paste commands instead.
How to disable. Quit the MLX Core app or stop the server started with `mlx-serve serve`. Remove the CLI with `brew uninstall mlx-serve` and the app with `brew uninstall --cask mlx-serve`. Launcher settings live in `~/.mlx-serve/`, and the fx entry in `~/.fx/settings.json` has to be removed by hand.
FAQ
How do I run Claude Code on a local model by hand?
Set `ANTHROPIC_BASE_URL='http://127.0.0.1:11234'`, an empty `ANTHROPIC_API_KEY`, `ANTHROPIC_AUTH_TOKEN=mlx-serve`, and put a model id from `GET /v1/models` into `ANTHROPIC_DEFAULT_OPUS_MODEL`, `ANTHROPIC_DEFAULT_SONNET_MODEL` and `ANTHROPIC_DEFAULT_HAIKU_MODEL`. Then run `claude --model` with the same id.
What does `mlx-serve run gemma4` do?
It downloads Gemma 4 E4B in 4-bit form, starts the server and opens a chat right in the terminal. Names can be short, `org/repo` from Hugging Face, or `name:tag`.
Which image and audio generators come with it?
By default FLUX.2-klein 4B in 4-bit, about 5 GB, makes images, LTX-Video 2.5 makes video, Qwen3-TTS does speech with voice cloning, ACE-Step 1.5 XL Turbo makes music and Hunyuan3D-2.1 builds 3D models. They are served at `/v1/images`, `/v1/audio`, `/v1/video` and `/v1/3d`.
Can an agent run in a sandbox?
In the MLX Core app one toggle turns on Agent Sandbox: every shell command from the agent runs in an isolated Linux virtual machine that boots in under a second.
How do I use another Mac's models?
The server shares models over the local network through Bonjour without setup. Even Claude Code pointed at `localhost` can then run on a 27-billion-parameter model from another Mac.
Is Python needed?
No. The server is a single Zig binary of about 7 MB, and the project's build uses no Python either.
Discussion
No questions or reviews yet.
Related
CLI toolsA local model server for Apple Silicon Macs: Qwen 3.8 and Gemma 4 with MTP decoding and OpenAI and Anthropic APIs for Claude Code, Codex and Cursor
LM Studio is a desktop app for running open models locally, with a chat, a local server in OpenAI and Anthropic formats, and MCP support.
The Apple Intelligence model built into a Mac as a terminal command and a local OpenAI-compatible server: no keys, no cloud
Ollama runs open models on your own machine and serves them to agents through a local API compatible with OpenAI and Anthropic.
Local coding agent with a terminal UI: it plans, drafts several candidates, tests them in a sandbox and repairs, so compact open models can handle code
A CLI and Python library for OpenAI, Claude, Gemini, local and other models: prompts, chats, tools, embeddings and a SQLite log
Comparison with similar tools
Entries that share the main search query, and how they differ.
| Entry | Official | Repository stars | Main difference |
|---|---|---|---|
| mlx-serve (this entry) | yes | 2,054 | |
| Ollama | yes | 184,341 | cross-platform runner for open models with an `ollama launch` command for agents, while mlx-serve runs on the MLX engine and adds its own app plus image, video and music generation |
| LM Studio | yes | an app with a window for running open models, chat and a local server, while mlx-serve ships as both a command line program and an app and also generates media | |
| MTPLX | yes | 2,627 | a local model server for Apple Silicon Macs |
README in short
The repository front page starts with installation: the app, Homebrew, Linux builds and building from source. Then come a comparison table with other engines, the feature list, a section on generating images, video, music, speech and 3D models, a description of the MLX Core app, supported models and speed measurements. Detailed documents sit in `docs/`: the API, the command line, agent integrations, models and performance. The repository has Zig and Swift sources, tests, a changelog and a Chinese translation.