agentperf-local CLI
agentperf-local measures how fast your machine serves an AI agent on recorded conversations: throughput and latency
- 132 ★ on the repository
- Last commit
- Apache-2.0
Minimal access
Why this level
- Downloads large model files and starts a local server on localhost but touches no project files or accounts
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add cli/agentperf-localDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewMinimal accessThe tool runs nothing and makes no network calls.
- Downloads large model files and starts a local server on localhost but touches no project files or accounts
This is third-party code. Review the repository files before installing.
What it does
A tool from Artificial Analysis that replays recorded agent conversations against an OpenAI-compatible model server and reports throughput and latency. Each request carries the full conversation so far, like a real agent's. It measures speed and not output quality. It can benchmark a server you already run, such as llama.cpp, LM Studio, vLLM, SGLang or Ollama, or download a pinned model, start the server and benchmark it. The default replay, `agentperf-default-v1`, is eight recorded agent tasks with 168 model turns under the `exact` output policy, so each turn generates its recorded number of tokens. `managed-run` downloads the model into the Hugging Face cache, checks every file's SHA-256, starts the server on localhost, runs the replay and stops it. A terminal UI walks you through choosing a model and running, and recipes live in `recipes/` by model and hardware.
Who it is for. People who run local models for Claude Code, Codex or OpenCode and choose between hardware, model and engine by real agent-workload speed.
Good fit when
- You are choosing between llama.cpp, vLLM and SGLang for a local model and want to compare them under an agent workload
- You want to see how your GPU or Mac copes with a long tool-using conversation
- Run results must be reproducible: the model is pinned by version and checked by SHA-256
- You want to benchmark an already running OpenAI-compatible server with no model downloads
Not a fit when
- You need an answer-quality score: the tool measures speed only
- Your model is hosted by a cloud provider: the scenario targets your own server and hardware
- You have no `uv` and cannot install Python 3.12: the tool runs through `uv`
Example request
Run agentperf-local managed-run for the qwen38-27b-q4-k-m profile on llama.cpp and show throughput and latencyLimitations
- `uv` is required and fetches Python 3.12 itself. Docker and Rust are optional. macOS, Linux and Windows are supported.
- The tool does not install serving frameworks. Recipes use `llama-server` on your PATH, vLLM or SGLang on NVIDIA GPUs, or Splash on Apple Silicon, with each version pinned in the recipe.
- On Windows managed runs need an NVIDIA GPU and llama.cpp, while vLLM and SGLang run only on Linux.
- Splash needs Apple Silicon M3 or newer and macOS 26.4 or newer, and the KV cache in that recipe is always eight-bit.
- A run downloads large model files into the Hugging Face cache and takes noticeable time.
How to disable. Remove the tool with `uv tool uninstall agentperf-local`. Downloaded models sit in the Hugging Face cache and results in the `--output-dir` folder, and both are deleted by hand.
FAQ
What does agentperf-local measure?
Agent serving speed: throughput and latency on recorded conversations. Answer quality is not scored.
How do I run it without installing?
Run `uvx agentperf-local`. For installation use `uv tool install agentperf-local` or `pipx install agentperf-local`.
What does managed-run do?
Downloads the pinned model into the Hugging Face cache, checks file SHA-256 sums, starts a server on localhost, runs the replay and stops the server.
What is in the default run?
`agentperf-default-v1`: eight recorded agent tasks and 168 model turns under the `exact` output policy.
Which engines are supported?
Recipes use llama.cpp, vLLM, SGLang and Splash on Apple Silicon. An already running OpenAI-compatible server such as LM Studio or Ollama can be benchmarked directly.
Discussion
No questions or reviews yet.
Related
CLI toolsA local-first AI agent running on your own machine via llama.cpp that drives a browser, edits files, runs approved commands and remembers context across session
Late is a coding AI agent in one Go binary: an orchestrator plans while disposable subagents code and research, on llama-server or cloud models
Agent Memory Benchmark runs agent memory systems over open datasets and scores accuracy, speed and token cost.
mi is agentic coding in 30 lines: a loop, two tools and a model; works with any OpenAI-compatible API and runs in Docker.
An open coding agent for the terminal, browser and desktop that works with models from OpenAI and Claude to Ollama
iFixAi probes an AI agent in simulated business situations, grades its replies and saves evidence of failures in reports.
README in short
The README states the task and a quick start through `uv`, then describes the default run, the `managed-run` mode with recipes and the terminal UI. A separate paragraph covers the Splash recipe for Apple Silicon.