glm-flash-lite skill
GLM-5.3-Flash on one 24 GB RTX 3090: the skill picks a mode, keeps experts in RAM and NVMe and starts an OpenAI-style server.
- 352 ★ on the repository
- Last commit
- MIT
Extended access
Why this level
- The skill starts Docker containers with GPU access and downloads about 125 GB of weights.
- Some steps need root or are hard to undo, and the skill asks for consent before them.
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add skills/glm-flash-liteDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- The skill starts Docker containers with GPU access and downloads about 125 GB of weights.
- Some steps need root or are hard to undo, and the skill asks for consent before them.
This is third-party code. Review the repository files before installing.
What it does
glm-flash-lite runs GLM-5.3-Flash (125 GB, EXL3 3.05 bpw) on one 24 GB GPU and keeps the experts that do not fit in VRAM in RAM and on NVMe. The server exposes an OpenAI-compatible API. Per the README, `nvme` mode with a 55 GiB memory cap reaches about 19.5 tokens per second for one user. The `glm53-offload-setup` skill guides the agent through setup: it detects hardware, picks a mode, builds and checks the expert store, starts the container, measures the standard table and tunes one setting at a time against a control run. The Docker image comes from `ghcr.io/sybil-solutions/glm53-flash-offload`.
Who it is for. Owners of a single 24 GB GPU and fast NVMe who want to run a large local model for agents without renting GPUs.
Good fit when
- An agent installs GLM-5.3-Flash on a home machine with an RTX 3090 and checks speed.
- A mode must be picked by available RAM: 16, 55 or 218 GiB.
- A local server must accept OpenAI-style requests for an agent client.
- Speed is below expectations and settings need tuning by measurement.
Not a fit when
- The GPU is not a 24 GB NVIDIA card, or there are several GPUs: the skill targets one.
- The model runs on a rented cloud GPU.
- There is no free disk space: about 125 GB for weights and 117 GB for the expert store.
Example request
Set up GLM-5.3-Flash on my machine with one RTX 3090: pick the mode from free memory, start the server and measure the speed table.Limitations
The skill targets Linux with one 24 GB NVIDIA card, and other cards or multi-GPU setups are out of scope. NVIDIA container toolkit, the `hf` CLI and fast local NVMe with xfs or ext4 are required. The 19.5 tokens per second figure was measured on an RTX 3090 with an EPYC 7443P and four NVMe drives in RAID0, and other hardware will differ. The `nvme-exact` mode is bit-exact at about 8 tokens per second. The last commit is from October 2026.
How to disable. Stop and remove the `glm53` container (`docker rm -f glm53`), then free the weights and expert-store directories. Remove the skill from the agent's skills directory.
FAQ
What address does the server use?
It is ready in about 80 seconds at `http://127.0.0.1:30000/v1`, and the model is named `glm-5.3-flash`.
How much memory is needed?
The `nvme` mode runs with a 55 GiB cap, and a 16 GiB variant is slower. The `fast` mode keeps everything in RAM and needs about 218 GiB.
How does `nvme-exact` differ from `nvme`?
It gives bit-exact output like stock exllamav3 but is slower: about 8.3 tokens per second versus 19.5.
Is there a way without Docker?
The README names the Omarchy Local AI plugin: `omarchy plugin add https://github.com/sybil-solutions/omarchy-local-ai --enable`.
Which cards does the skill suit?
One NVIDIA RTX 3090 with 24 GB. Recipes for other cards such as DGX Spark or RTX PRO 6000 use other engines and images.
Discussion
No questions or reviews yet.
Related
SkillsTen skills for LLM serving: SGLang and vLLM benchmarks, profiler triage, memory planning and PR history.
A local Mac server with faster generation and identical output: Claude Code and Codex run on a model on your own computer
Train in Silence estimates the VRAM a fine-tuning job needs and ranks cloud GPU offers by cost and run time.
llmman runs Claude Code, Codex and OpenCode on a local or hosted model with one command and stores models as OCI images
A skill for Claude Code and Gemini CLI that runs LLM fine-tuning from dataset to GGUF export, on NVIDIA GPUs and Apple Silicon
A local GGUF model server accepts OpenAI-style requests so applications can be tested on hardware you control.
README in short
The README opens with a speed table by mode, then gives a run via the Omarchy plugin or Docker and tables of verified recipes for the RTX 3090 with status and image digest fields.