https://github.com/openml/benchmark-suites
https://github.com/openml/benchmark-suites
Last synced: 9 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/openml/benchmark-suites
- Owner: openml
- Created: 2017-11-10T15:42:05.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2021-09-04T22:51:45.000Z (almost 5 years ago)
- Last Synced: 2025-07-01T22:36:05.679Z (12 months ago)
- Language: Jupyter Notebook
- Size: 5.16 MB
- Stars: 7
- Watchers: 6
- Forks: 3
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# OpenML Benchmark Suites
Machine learning research depends on objectively interpretable, comparable, and reproducible algorithm benchmarks. Therefore, we advocate the use of curated, comprehensive suites of machine learning datasets to standardize the setup, execution, and reporting of benchmarks. We enable this through platform-independent software tools that help to create and leverage these benchmarking suites. These are seamlessly integrated into the OpenML platform, and accessible through interfaces in Python, Java, and R.
OpenML benchmarking suites are:
- easy to use through standardized data formats, APIs, andclient libraries
- machine-readable, with extensive meta-information on the includeddatasets
- allow benchmarks to be shared and reused in future studies.
## Documentation
[Detailed documentation on how to create and use OpenML benchmark suites](https://docs.openml.org/benchmark)
This also includes a list of current benchmark suites, such as the OpenML-CC18.
## Notebooks
We provide a set of notebooks to explore existing benchmark suites, and create your own:
- Automated benchmark suite generator: Allows you to specific a list of constraints and additional tests, and retrieve all datasets that adhere to them
- CC18 score overview: Overview of shared results on the CC18 benchmark suites
- CC18 benchmark analysis: A deeper analysis of existing results in R (note: this was done for an older benchmark set)
- Mini-Benchmark of R algorithm on the CC18: http://rpubs.com/giuseppec/OpenML100
- Mini-Benchmark of WEKA algorithms on the CC18
- Tutorials for OpenML in R and Python