{"id":15973411,"url":"https://github.com/optuna/kurobako","last_synced_at":"2025-10-12T15:40:41.714Z","repository":{"id":38331312,"uuid":"175211373","full_name":"optuna/kurobako","owner":"optuna","description":"A black-box optimization benchmark tool","archived":false,"fork":false,"pushed_at":"2023-06-21T22:41:28.000Z","size":836,"stargazers_count":80,"open_issues_count":10,"forks_count":9,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-05-01T11:35:05.519Z","etag":null,"topics":["black-box-benchmarking","black-box-optimization","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/optuna.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-12T12:56:25.000Z","updated_at":"2024-06-25T01:53:30.676Z","dependencies_parsed_at":"2024-06-25T01:53:27.999Z","dependency_job_id":"e6a34658-675b-4171-9d13-209f0a177a78","html_url":"https://github.com/optuna/kurobako","commit_stats":{"total_commits":549,"total_committers":4,"mean_commits":137.25,"dds":0.02732240437158473,"last_synced_commit":"83ae97e85c64a4260c31ee32851ee4e80a7b6052"},"previous_names":[],"tags_count":41,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/optuna%2Fkurobako","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/optuna%2Fkurobako/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/optuna%2Fkurobako/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/optuna%2Fkurobako/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/optuna","download_url":"https://codeload.github.com/optuna/kurobako/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248018060,"owners_count":21034048,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["black-box-benchmarking","black-box-optimization","rust"],"created_at":"2024-10-07T21:04:49.509Z","updated_at":"2025-10-12T15:40:36.659Z","avatar_url":"https://github.com/optuna.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"kurobako\n=========\n\n[![kurobako](https://img.shields.io/crates/v/kurobako.svg)](https://crates.io/crates/kurobako)\n[![Documentation](https://docs.rs/kurobako/badge.svg)](https://docs.rs/kurobako)\n[![Actions Status](https://github.com/optuna/kurobako/workflows/CI/badge.svg)](https://github.com/optuna/kurobako/actions)\n[![Coverage Status](https://coveralls.io/repos/github/optuna/kurobako/badge.svg?branch=master)](https://coveralls.io/github/optuna/kurobako?branch=master)\n[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)\n[![Docker Hub](https://img.shields.io/badge/docker-ready-blue.svg)](https://hub.docker.com/r/sile/kurobako)\n\n`kurobako` is a command-line tool to benchmark black-box optimization algorithms.\n\nFeatures:\n- It has the capability to support wide range of optimization problems:\n  - various search sapce:\n    - Continuous numerical, discrete numerical and categorical\n    - Uniform distribution and log uniform distribution\n    - Conditional\n  - Constrainted problems\n  - Multi-objective problems\n- Generating a markdown report and PNG plots from benchmarking results\n- Easy to add user-defined optimization problems and solvers\n- Simulating a concurrent environment in which an optimization process is executed by multiple workers simultaneously\n- Reproducible\n\n\nInstallation\n------------\n\n### Precompiled binaries\n\nPrecompiled binaries for Linux are available in the [releases] page.\n\n```console\n$ curl -L https://github.com/optuna/kurobako/releases/download/${VERSION}/kurobako-${VERSION}.linux-amd64 -o kurobako\n$ chmod +x kurobako\n$ ./kurobako -h\n```\n\n[releases]: https://github.com/optuna/kurobako/releases\n\n### Using Cargo\n\nIf you have already installed [Cargo][cargo], you can install `kurobako` by executing the following command:\n\n```console\n$ cargo install kurobako\n```\n\n[cargo]: https://doc.rust-lang.org/cargo/\n\n### Dependencies\n\nIf you want to use `kurobako plot` command, you need to have installed `gnuplot` package on your environment.\n\nUsage Example\n--------------\n\n```console\n// Define solver.\n$ kurobako solver random | tee solver.json\n{\"random\":{}}\n\n// Define problem.\n$ curl -OL http://ml4aad.org/wp-content/uploads/2019/01/fcnet_tabular_benchmarks.tar.gz\n$ tar xf fcnet_tabular_benchmarks.tar.gz \u0026\u0026 cd fcnet_tabular_benchmarks/\n$ kurobako problem hpobench fcnet_protein_structure_data.hdf5 | tee problem.json\n{\"hpobench\":{\"dataset\":\"fcnet_protein_structure_data.hdf5\"}}\n\n// Run benchmark.\n$ kurobako studies --solvers $(cat solver.json) --problems $(cat problem.json) | kurobako run \u003e result.json\n(ALL) [00:00:01] [STUDIES     10/10 100%] [ETA  0s] done\n\n// Report the benchmark result.\n$ cat result.json | kurobako report\n...abbrev...\n\n// Plot the benchmark result.\n$ cat result.json | kurobako plot curve\n(PLOT) [00:00:01] [1/1 100%] [ETA  0s] done (dir=\"images/curve/\")\n```\n\nBuild-in Solvers and Problems\n-----------------------------\n\nSolvers:\n- Random Search\n- [NSGA-II](https://ieeexplore.ieee.org/document/996017)\n- [ASHA](https://arxiv.org/abs/1810.05934)\n- [Optuna](https://github.com/optuna/optuna)\n\nProblems:\n- [NASBench](https://github.com/automl/nas_benchmarks) ([detail](https://github.com/optuna/kurobako/wiki/NASBench))\n- [HPOBench](https://github.com/automl/nas_benchmarks)\n- [sigopt/evalset](https://github.com/sigopt/evalset)\n- [Two-objective ZDT functions](http://repository.ias.ac.in/9404/1/306.pdf)\n\nWhere does the name come from?\n-----------------------------------\n\n\"kurobako\" is a Japanese translation of \"black box\".\n\n\nReferences\n----------\n\n- [Introduction to Kurobako: A Benchmark Tool for Hyperparameter Optimization Algorithms](https://medium.com/optuna/kurobako-a2e3f7b760c7)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foptuna%2Fkurobako","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foptuna%2Fkurobako","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foptuna%2Fkurobako/lists"}