Glyd CLI
Glyd’s CLI runs language models on NVIDIA GPUs with compressed weights and cache to reduce the memory needed for inference.
- 231 ★ on the repository
- Last commit
Extended access
Why this level
- The installer and model run locally and consume disk space and GPU resources.
Install
Manual install
curl -LsSf https://getglyd.com/install.sh | shFirst command in the model-running section. Review the script first; the Homebrew codec installation serves a different task.
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- The installer and model run locally and consume disk space and GPU resources.
This is third-party code. Review the repository files before installing.
What it does
Glyd launches a model from the terminal and selects settings for the available GPU. Its main mode stores weights and attention cache compactly, reconstructing the original bf16 values during computation. Freed GPU memory can support more context or additional requests. The project also integrates with vLLM and Python. A separate file codec is available, but model work uses the GPU package. More aggressive compression modes reduce weight precision.
Who it is for. Local inference engineers short on GPU memory for a selected bf16 model or its context window.
Good fit when
- A model nearly fits GPU memory and you want to test compressed weights.
- A team serving through vLLM wants to evaluate additional cache capacity.
- You need a test chat on your GPU without writing a Python application.
- An experiment compares lossless weights with more aggressive compression modes.
Not a fit when
- The target is a Mac without a supported NVIDIA GPU.
- The workload must run entirely on CPU.
- A commercial project cannot arrange licensing with the author.
Example request
Check my Linux server’s Glyd compatibility and prepare a Qwen test measuring memory before and after launch.Limitations
GPU inference requires Linux x86_64 or aarch64, an NVIDIA Ampere, Ada, Hopper or Blackwell GPU and driver 580 or newer. Lossless compression preserves weights and cache values; accumulation order can still affect final output bits unless exact mode is selected. The kestrel and swift profiles reduce precision and need separate quality evaluation. From version 0.29.0, BUSL-1.1 applies and commercial use requires a separate license.
How to disable. Stop the Glyd process or vLLM server. Preserve model settings and check that other applications no longer depend on it before changing environments.
FAQ
How do I open a model chat?
After installation, the guide uses `glyd run Qwen/Qwen3.5-9B`. The CLI selects GPU settings and opens a terminal chat.
Can I keep vLLM?
Yes. The project provides `glyd[vllm]` and a Glyd quantization option. Check version compatibility before load testing.
Is the memory saving always the same?
No. It depends on the model and hardware. Measure your workload using the published benchmark procedures.
Is the Homebrew command the GPU-model installer?
That section installs the file codec. For inference, use the model-running section and its Linux and GPU requirements.
Where can I inspect benchmark evidence?
The benchmarks directory and project results page contain run logs. Match the model and GPU to your environment.
Discussion
No questions or reviews yet.
Related
CLI toolsA GPU profiler for AI agents: the agent runs vLLM or SGLang under graphsignal-run, reads the metrics and tunes flags for speed
A CLI and MCP server for the Qizhi platform: free GPU nodes, job submission and logs, queue reasons from Claude Code
A self-hosted gateway for multiple LLM keys and providers: rotation, failover, request logs and usage accounting
A local Rust proxy that compresses the model API requests of Claude Code, Codex and Cursor so you pay for fewer tokens
The official Vast.ai skill: an agent searches GPU offers, rents instances and manages them through vastai.
The CLI starts a language-model proxy and a request tracking service for debugging applications across model providers.
README in short
Repository measurements identify models and environments across several NVIDIA GPUs. Results distinguish weight size from available context capacity.