Soup CLI
A CLI for fine-tuning language models from one YAML: LoRA, QLoRA, DPO and GRPO, layer streaming for 8B on a 4 GB GPU, export to GGUF
- 9,577 ★ on the repository
- Last commit
- Apache-2.0
Broad access
Why this level
- It runs training, installs packages and downloads weights from Hugging Face; a GGUF export may call pip unless `--no-install-deps` is given
- `soup push` and `--push-as` send models and checkpoints to your Hugging Face account with a token
- Running `--cloud modal` or `--cloud lambda` only prepares a plan, while `--cloud-submit` sends a paid job to the cloud
Install
In your terminal, with SkillFoxx CLI
npx skillfoxx add cli/soupDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- It runs training, installs packages and downloads weights from Hugging Face; a GGUF export may call pip unless `--no-install-deps` is given
- `soup push` and `--push-as` send models and checkpoints to your Hugging Face account with a token
- Running `--cloud modal` or `--cloud lambda` only prepares a plan, while `--cloud-submit` sends a paid job to the cloud
This is third-party code. Review the repository files before installing.
What it does
Soup is the `soup` command-line utility for fine-tuning language models: you describe the job in a single `soup.yaml`, and it picks the batch size, finds the GPU and turns on quantization. It supports supervised fine-tuning (SFT), the pairwise methods DPO, KTO, ORPO, SimPO and IPO, and GRPO and PPO. Any Hugging Face model that loads through `AutoModelForCausalLM` can serve as the base; ready recipes exist for Llama 3.x and 4, Qwen 2.5 and 3, Gemma 3, Mistral, DeepSeek and Phi-4, and `soup recipes list` prints them. The typical path is short. `soup init --template chat` creates a config, `soup train --config soup.yaml` starts training, `soup chat --model ./output` opens a dialogue with the result, `soup merge --adapter ./output` folds the LoRA adapter into the base model, and `soup export --model ./output --format gguf --quant q4_k_m` prepares a file for Ollama and llama.cpp. The remaining commands cover batch inference, `soup serve` with an OpenAI-compatible server, quality evaluation and data checks; `soup ui` opens a local dashboard in the browser. One feature is aimed at people who build agents and is called Agent Forge. The `soup agent synth --spec api.yaml` command parses an OpenAPI description, an MCP server manifest or a GraphQL schema and builds a set of tool-calling examples for training with no manual labelling. Another distinctive feature, layer streaming, feeds the frozen base to the GPU one layer at a time, and by the author's measurements Llama-3.1-8B with NF4 fits in a 3.32 GB peak on a 4 GB laptop RTX 3050.
Who it is for. For developers and researchers who fine-tune open models on their own GPU and want one config instead of assembling a training stack by hand.
Good fit when
- You need to fine-tune a 7–8B model with QLoRA on a home GPU with 8 GB of memory or on a laptop without writing a training script yourself
- You have a set of examples in Alpaca, ShareGPT or ChatML format, or DPO pairs, and want an adapter from one command
- A finished model must be exported to GGUF and deployed to Ollama right away with `soup export --deploy ollama`
- You want to check data and training before spending GPU hours: `soup doctor`, `soup data inspect` and `soup advise` give a preliminary assessment
- An agent that calls tools needs its own training examples built from an OpenAPI description or an MCP server manifest
Not a fit when
- You run Python 3.13 or newer: the package declares support for 3.10–3.12 only, since PyTorch builds on newer versions were not tested
- You only want to run a finished model with no training: Ollama is enough, while Soup is for when a model is fine-tuned
- You have no GPU at all: every task runs on the CPU too, but the author calls that mode experimental and very slow
- Your config contains options such as `quantization_aware: true` or `use_longlora`: since 0.76.0 such settings, which used to be accepted and never applied, stop the load
Example request
Install soup-cli with the train extra, create a config from the chat template for Llama-3.1-8B-Instruct on my train.jsonl and start QLoRA fine-tuningLimitations
Version 0.76.0 is marked alpha in the package metadata. The light `soup-cli` install without PyTorch can only create configs and check data, and training needs the additional `soup-cli[train]` set.
Layer streaming is switched on with `stream_layers: true` and the author calls it a beta feature. The figures of 119.6 tokens per second and a 3.32 GB peak were taken before the correctness fix in version 0.73.0, and another contributor's re-measurement gave 208.6 tokens per second at a 2.40 GB peak on a different clock; the author warns that these rows are not comparable.
Since version 0.75 unknown config keys cause a load error rather than being silently dropped. The AWQ and GPTQ export formats are deprecated and will disappear in version 0.77: both source projects are archived.
Soup sends usage statistics only after you explicitly enable `SOUP_TELEMETRY=1`.
How to disable. Remove the utility with `pipx uninstall soup-cli` or `pip uninstall soup-cli`. Training results live in the directory from the `output` field of your config and service files in `~/.soup/`; delete them by hand.
FAQ
How do I fine-tune a model on a GPU with 4 GB of memory?
Turn on `stream_layers: true` in the config. The frozen base then sits in RAM or on NVMe and is fed to the GPU one decoder layer at a time. The author measured Llama-3.1-8B-Instruct with NF4 on an RTX 3050 Laptop 4 GB: a 3.32 GB memory peak. The feature is beta, and you can check the claim in the `notebooks/proof-4gb.ipynb` notebook on a free Colab T4.
Which Python versions does soup-cli support?
Only 3.10, 3.11 and 3.12: `pyproject.toml` carries a `<3.13` bound. On newer versions the package manager installed untested PyTorch builds that crashed in a native extension before Soup itself started.
How does the light install differ from soup-cli[train]?
Plain `pip install soup-cli` does not pull PyTorch and gives the `soup init` and `soup data` commands and checks. The `[train]` set adds torch, transformers, peft, trl, datasets, bitsandbytes and accelerate. The web dashboard needs `[ui]`, and `[all]` installs training, the server, the dashboard and data tools at once.
How do I export a trained model to Ollama?
`soup export --model ./output --format gguf --quant q4_k_m` prepares a GGUF, and `soup export --model ./output --deploy ollama` exports and deploys the file to Ollama right away. Separately, `soup deploy ollama --model m.gguf --name x` exists, `--list` shows models deployed by Soup and `--remove` deletes them.
What is Agent Forge and what is it for?
These are the `soup agent synth`, `soup agent train` and `soup agent eval` commands. They read OpenAPI 3.x, an MCP server manifest or GraphQL introspection and build training examples of a user request and a tool call. `$ref` links in OpenAPI stay unresolved, so files are not read from outside paths.
Why did my old config stop loading after an update?
Since version 0.75 unknown keys raise an error, and in 0.76.0 options that were accepted but read by nothing also stop the load. Run `soup doctor --config soup.yaml` before launching: the message names the field and what to change it to.
Discussion
No questions or reviews yet.
Related
CLI toolsA skill and Python package for hyperparameter tuning: the agent reads the code, proposes the next set of values, and your own run measures the result
NVIDIA's skill for Claude Code and Codex: the agent turns a dataset description into a config, validates it, previews it and generates synthetic data
Official Hugging Face skills: Hub operations via the hf CLI, datasets, model training, Spaces, evals and deployment
A graph database with git-style branches: the server exposes it to agents over MCP at /mcp, and a skill teaches the agent schemas and queries
On-premises RAG in containers with an MCP server: the agent searches your documents while data stays local
A three-stage workflow for math modeling tasks: analysis and model choice, Python or MATLAB computation, and paper assembly in Word or LaTeX
README in short
The project description opens with the promise of fine-tuning models in one command without setting up infrastructure and shows layer streaming with measurements on laptop GPUs. It continues with the highlights of version 0.76.0 including breaking changes, installation through pipx, uv and pip, config templates, a full `soup.yaml` example, a list of common commands, a table of model size by video memory and running in Docker. The repository holds detailed guides on training, data, evaluation, export and the privacy policy, example configs, a catalog of measurements with reproducible scripts and tests; the description is translated into Turkish, Arabic, Japanese and Chinese.