optim-agent skill
A skill and Python package for hyperparameter tuning: the agent reads the code, proposes the next set of values, and your own run measures the result
- 797 ★ on the repository
- Last commit
- MIT
Broad access
Why this level
- Runs your evaluation command many times: for model training that means hours of compute and money
- Writes trial logs to the project's .optim-agent-runs folder and appends it to .git/info/exclude
- In AgentSampler mode launches an installed command-line agent for every trial
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add skills/optim-agentDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- Runs your evaluation command many times: for model training that means hours of compute and money
- Writes trial logs to the project's .optim-agent-runs folder and appends it to .git/info/exclude
- In AgentSampler mode launches an installed command-line agent for every trial
This is third-party code. Review the repository files before installing.
What it does
optim-agent tunes the parameters of a system against a measurable goal, with a coding agent doing the choosing: Claude Code, Codex or OpenCode. It fits when a system has configurable parameters and a number to raise or lower: model quality, latency, cost, a strategy's return. In skill mode the agent reads the entry point of the evaluation and the files that define the parameters, records the type, legal range and meaning of each, and then works in a loop. The loop goes like this. First the agent confirms the goal, direction, trial budget, evaluation command and time and cost limit, then measures the starting configuration with the same command. Next it creates a study with `oa.create_study` and storage in `.optim-agent-runs/skill-study.json`, picks values from the code and the full history, takes a trial with `study.ask(params)` and reports through `study.tell(trial, value)`. A crashed trial is marked failed, a deliberately stopped one pruned. For noisy evaluations the skill repeats promising points before declaring a winner. When the budget ends the skill reports the baseline, the best value and parameters, the trial count, failed and pruned trials, the convergence trend and a reproduction command. The same package also works without the skill: `oa.AgentSampler(backend="claude")` calls a command-line agent itself and asks for a proposal in every trial.
Who it is for. For machine learning engineers, researchers and service developers for whom evaluating one configuration is expensive and trying values blindly is not an option.
Good fit when
- You need to tune learning rate, batch size, regularization and other training parameters, and each run takes a long time
- You are tuning model serving: quantization, batching, caching or routing for latency and cost
- You need to fit the parameters, thresholds and rebalancing rules of a quantitative strategy to a metric on history
- You have a script that prints one number for a set of parameters and want an auditable log of every trial
Not a fit when
- There is no measurable goal: a trial's result must be a number from your evaluation command, and the skill neither invents nor adjusts values itself
- You want to replace a reinforcement learning algorithm: the author states that optim-agent tunes the system around the learning loop and does not train the policy
- No authenticated command-line agent: `AgentSampler` mode needs one of `claude`, `codex` or `opencode` on PATH
- A single trial is instant and cheap: classical methods such as random search will do without the cost of agent calls
Example request
Use optim-agent to tune the learning rate and batch size for train.py, minimizing val_loss with a budget of 20 trialsLimitations
The package is version 0.2.0, marked Alpha in its metadata, needs Python 3.9 or newer and uses the MIT license.
The skill runs your evaluation command on every trial, so cost and time are set by it: fix the trial budget and the cost limit before the first run. Evaluation runs must not touch the production configuration, so keep them separate.
Sequential trials are the default because each proposal uses the whole earlier history. Several processes need SQLite storage (`skill-study.db`). Trial files live in `.optim-agent-runs/`, and on first run the skill appends that folder to `.git/info/exclude`.
The skill does not tune secrets, credentials or unbounded parameters. The author's comparisons with TPE and random search were run on the Branin and Ackley functions, MNIST, CIFAR-10 and two Gymnasium environments: such a gain on your tasks is not guaranteed.
How to disable. For Claude Code disable the plugin through `/plugin`; for Codex delete the skill folder from the skills directory. `python -m pip uninstall optim-agent` removes the package. Trial logs stay in the project's `.optim-agent-runs/` and you remove them yourself.
FAQ
How does tuning by an agent differ from Optuna and random search?
The agent relies on the meaning of the parameters, which you describe in `context` hints, and on trial history, not on numbers alone. The author compares the approach with TPE and random search on several tasks and stresses that without hints the result is worse.
What happens if the agent proposes a value outside the range?
The package validates the proposal against the declared parameter space and falls back to safe sampling when the agent's reply is invalid. The agent only proposes values, and the measured result stays decisive.
How do I resume a study after an interruption?
JSON storage records a trial the moment `study.tell` runs. Before an expensive run the skill saves the parameters, command and output path in the trial folder, and after an interruption checks the output and either records the finished result or reruns the trial.
Which agents are supported?
In skill mode, Claude Code, Codex, OpenCode and OpenClaw, or any agent that reads project files and runs shell commands. `AgentSampler` takes the `claude`, `codex` or `opencode` backend.
Do I need a paid model?
`AgentSampler` needs one authenticated command-line agent. The author separately measured the models in the OpenCode pool marked free and notes that this pool rotates.
Discussion
No questions or reviews yet.
Related
SkillsHigh-Stakes Analytics Lab
High-Stakes Analytics & Decision Lab
An evidence-gated analytics skill for high-stakes decisions: profiles data, picks a method the evidence supports, can answer with a data request
An agent skill that turns an arxiv paper into a minimal Python implementation, with code anchored to sections and equations and gaps flagged honestly
Auto-Empirical Research Skills
Auto-Empirical Research Skills (AERS)
A plugin marketplace for empirical social science research: causal inference, econometrics and manuscript prep for top journals
166 skills for scientific work: bioinformatics, cheminformatics, clinical data, geospatial analysis and 100+ databases
SciPilot Figure
scipilot-figure-skill
A scientific visualization advisor skill: it profiles the data and picks a chart for the argument first, then renders it to journal-grade quality
A library of agent skills for medical research: literature search and appraisal, study design, data analysis and academic writing
README in short
The project description in English is translated into eight more languages, Russian among them. It has installation routes, a Python example, a table of application areas and detailed comparisons with TPE and random search: the Branin and Ackley functions, the MNIST and CIFAR-10 classifiers and Q-learning controllers. The repository also holds examples, tests, reproducible benchmarks, a paper and a roadmap.
README badge
Original SKILL.md
--- name: optim-agent description: Use when the user wants to optimize configurable system parameters against a measurable scalar objective, especially for model training, inference, quantitative strategies, reinforcement learning, scientific workflows, or other expensive black-box evaluations where reading the project can improve trial selection. --- # optim-agent Act as the sampler inside any coding-agent session: Claude Code, Codex, OpenCode/OpenClaw, or another agent that can read project files and run shell commands. Read the project to understand parameter meaning and interactions, propose one configuration, run the real evaluator, and record the result through optim-agent's ask/tell API. Let the measured objective, not the agent's intuition, decide what works. ## Load the workflow Use this file as the operating guide for the active coding agent. In Codex, it can be installed directly from GitHub: ```text $skill-installer install https://github.com/Optim-Agent/optim-agent ``` In Claude Code, OpenCode/OpenClaw, or another coding-agent environment, place this repository or `SKILL.md` in the agent-visible workspace and ask the agent to follow the optim-agent workflow. The workflow does not depend on Codex-only APIs; it needs file access, shell access, and Python. Ensure the Python package is importable. Choose one source; do not install both: