{"id":13688833,"url":"https://github.com/benchopt/benchopt","last_synced_at":"2025-05-15T07:05:21.938Z","repository":{"id":37090107,"uuid":"227720375","full_name":"benchopt/benchopt","owner":"benchopt","description":"Making your benchmark of optimization algorithms simple and open","archived":false,"fork":false,"pushed_at":"2025-05-06T17:36:48.000Z","size":2138,"stargazers_count":262,"open_issues_count":141,"forks_count":64,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-05-06T18:42:37.250Z","etag":null,"topics":["benchmark","julia-language","machine-learning","optimization","optimization-methods","python","rlang"],"latest_commit_sha":null,"homepage":"https://benchopt.github.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/benchopt.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"roadmap.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-12-13T00:15:23.000Z","updated_at":"2025-05-06T17:36:52.000Z","dependencies_parsed_at":"2024-01-17T06:07:41.826Z","dependency_job_id":"9e264e0d-662d-444d-abdb-5bc98cb4eb1b","html_url":"https://github.com/benchopt/benchopt","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benchopt%2Fbenchopt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benchopt%2Fbenchopt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benchopt%2Fbenchopt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benchopt%2Fbenchopt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/benchopt","download_url":"https://codeload.github.com/benchopt/benchopt/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254292040,"owners_count":22046426,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","julia-language","machine-learning","optimization","optimization-methods","python","rlang"],"created_at":"2024-08-02T15:01:24.245Z","updated_at":"2025-05-15T07:05:16.925Z","avatar_url":"https://github.com/benchopt.png","language":"Python","readme":".. image:: https://raw.githubusercontent.com/benchopt/communication_materials/main/posters/images/logo_benchopt.png\n   :width: 350\n   :align: center\n\n*—Making your ML and optimization benchmarks simple and open—*\n\n----\n\n|Test Status| |codecov| |Documentation| |Python 3.6+| |install-per-months| |discord| |SWH|\n\n``Benchopt`` is a benchmarking suite tailored for machine learning workflows.\nIt is built for simplicity, transparency, and reproducibility.\nIt is implemented in Python but can run algorithms written in **many programming languages**.\n\n\nSo far, ``benchopt`` has been tested with `Python \u003chttps://www.python.org/\u003e`_,\n`R \u003chttps://www.r-project.org/\u003e`_, `Julia \u003chttps://julialang.org/\u003e`_\nand `C/C++ \u003chttps://isocpp.org/\u003e`_ (compiled binaries with a command line interface).\nPrograms available via `conda \u003chttps://docs.conda.io/en/latest/\u003e`_ should be compatible as well.\nSee for instance an `example of usage \u003chttps://benchopt.github.io/auto_examples/plot_run_benchmark_python_R.html\u003e`_ with ``R``.\n\n\nInstall\n-------\n\nIt is recommended to use ``benchopt`` within a ``conda`` environment to fully-benefit\nfrom ``benchopt`` Command Line Interface (CLI).\n\n\nTo install ``benchopt``, start by creating a new ``conda`` environment and then activate it\n\n.. code-block:: bash\n\n    conda create -n benchopt python\n    conda activate benchopt\n\nThen run the following command to install the **latest release** of ``benchopt``\n\n.. code-block:: bash\n\n    pip install -U benchopt\n\nIt is also possible to use the **latest development version**. To do so, run instead\n\n.. code-block:: bash\n\n    pip install git+https://github.com/benchopt/benchopt.git\n\n\nGetting started\n---------------\n\nAfter installing ``benchopt``, you can\n\n- replicate/modify an existing benchmark\n- create your own benchmark\n\n\nUsing an existing benchmark\n^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nReplicating an existing benchmark is simple.\nHere is how to do so for the `L2-logistic Regression benchmark \u003chttps://github.com/benchopt/benchmark_logreg_l2\u003e`_.\n\n1. Clone the benchmark repository and ``cd`` to it\n\n.. code-block:: bash\n\n   git clone https://github.com/benchopt/benchmark_logreg_l2\n   cd benchmark_logreg_l2\n\n2. Install the desired solvers automatically with ``benchopt``\n\n.. code-block:: bash\n\n   benchopt install . -s lightning -s sklearn\n\n3. Run the benchmark to get the figure below\n\n.. code-block:: bash\n\n   benchopt run . --config ./example_config.yml\n\n\n.. figure:: https://benchopt.github.io/_images/sphx_glr_plot_run_benchmark_001.png\n   :target: how.html\n   :align: center\n   :width: 60%\n\nThese steps illustrate how to reproduce the `L2-logistic Regression benchmark \u003chttps://github.com/benchopt/benchmark_logreg_l2\u003e`_.\nFind the complete list of the `Available benchmarks`_.\nAlso, refer to the `documentation \u003chttps://benchopt.github.io/\u003e`_ to learn more about ``benchopt`` CLI and its features.\nYou can also easily extend this benchmark by adding a dataset, solver or metric.\nLearn that and more in the `Benchmark workflow \u003chttps://benchopt.github.io/benchmark_workflow/index.html\u003e`_.\n\n\nCreating a benchmark\n^^^^^^^^^^^^^^^^^^^^\n\nThe section `Write a benchmark \u003chttps://benchopt.github.io/benchmark_workflow/write_benchmark.html\u003e`_ of the documentation provides a tutorial\nfor creating a benchmark. The ``benchopt`` community also maintains\na `template benchmark \u003chttps://github.com/benchopt/template_benchmark\u003e`_ to quickly and easily start a new benchmark.\n\n\nFinding help\n------------\n\nJoin ``benchopt`` `discord server \u003chttps://discord.gg/EA2HGQb7nv\u003e`_ and get in touch with the community!\nFeel free to drop us a message to get help with running/constructing benchmarks\nor (why not) discuss new features to be added and future development directions that ``benchopt`` should take.\n\n\nCiting Benchopt\n---------------\n\n``Benchopt`` is a continuous effort to make reproducible and transparent ML and optimization benchmarks.\nJoin us in this endeavor! If you use ``benchopt`` in a scientific publication, please cite\n\n.. code-block:: bibtex\n\n   @inproceedings{benchopt,\n      author    = {Moreau, Thomas and Massias, Mathurin and Gramfort, Alexandre\n                   and Ablin, Pierre and Bannier, Pierre-Antoine\n                   and Charlier, Benjamin and Dagréou, Mathieu and Dupré la Tour, Tom\n                   and Durif, Ghislain and F. Dantas, Cassio and Klopfenstein, Quentin\n                   and Larsson, Johan and Lai, En and Lefort, Tanguy\n                   and Malézieux, Benoit and Moufad, Badr and T. Nguyen, Binh and Rakotomamonjy,\n                   Alain and Ramzi, Zaccharie and Salmon, Joseph and Vaiter, Samuel},\n      title     = {Benchopt: Reproducible, efficient and collaborative optimization benchmarks},\n      year      = {2022},\n      booktitle = {NeurIPS},\n      url       = {https://arxiv.org/abs/2206.13424}\n   }\n\n\nAvailable benchmarks\n--------------------\n\n.. list-table::\n   :widths: 70 15 15\n   :header-rows: 1\n\n   * - Problem\n     - Results\n     - Build Status\n   * - `Ordinary Least Squares (OLS) \u003chttps://github.com/benchopt/benchmark_ols\u003e`_\n     - `Results \u003chttps://benchopt.github.io/results/benchmark_ols.html\u003e`__\n     - |Build Status OLS|\n   * - `Non-Negative Least Squares (NNLS) \u003chttps://github.com/benchopt/benchmark_nnls\u003e`_\n     - `Results \u003chttps://benchopt.github.io/results/benchmark_nnls.html\u003e`__\n     - |Build Status NNLS|\n   * - `LASSO: L1-Regularized Least Squares \u003chttps://github.com/benchopt/benchmark_lasso\u003e`_\n     - `Results \u003chttps://benchopt.github.io/results/benchmark_lasso.html\u003e`__\n     - |Build Status Lasso|\n   * - `LASSO Path \u003chttps://github.com/jolars/benchmark_lasso_path\u003e`_\n     - `Results \u003chttps://benchopt.github.io/results/benchmark_lasso_path.html\u003e`__\n     - |Build Status Lasso Path|\n   * - `Elastic Net \u003chttps://github.com/benchopt/benchmark_elastic_net\u003e`_\n     -\n     - |Build Status ElasticNet|\n   * - `MCP \u003chttps://github.com/benchopt/benchmark_mcp\u003e`_\n     - `Results \u003chttps://benchopt.github.io/results/benchmark_mcp.html\u003e`__\n     - |Build Status MCP|\n   * - `L2-Regularized Logistic Regression \u003chttps://github.com/benchopt/benchmark_logreg_l2\u003e`_\n     - `Results \u003chttps://benchopt.github.io/results/benchmark_logreg_l2.html\u003e`__\n     - |Build Status LogRegL2|\n   * - `L1-Regularized Logistic Regression \u003chttps://github.com/benchopt/benchmark_logreg_l1\u003e`_\n     - `Results \u003chttps://benchopt.github.io/results/benchmark_logreg_l1.html\u003e`__\n     - |Build Status LogRegL1|\n   * - `L2-regularized Huber regression \u003chttps://github.com/benchopt/benchmark_huber_l2\u003e`_\n     -\n     - |Build Status HuberL2|\n   * - `L1-Regularized Quantile Regression \u003chttps://github.com/benchopt/benchmark_quantile_regression\u003e`_\n     - `Results \u003chttps://benchopt.github.io/results/benchmark_quantile_regression.html\u003e`__\n     - |Build Status QuantileRegL1|\n   * - `Linear SVM for Binary Classification \u003chttps://github.com/benchopt/benchmark_linear_svm_binary_classif_no_intercept\u003e`_\n     -\n     - |Build Status LinearSVM|\n   * - `Linear ICA \u003chttps://github.com/benchopt/benchmark_linear_ica\u003e`_\n     -\n     - |Build Status LinearICA|\n   * - `Approximate Joint Diagonalization (AJD) \u003chttps://github.com/benchopt/benchmark_jointdiag\u003e`_\n     -\n     - |Build Status JointDiag|\n   * - `1D Total Variation Denoising \u003chttps://github.com/benchopt/benchmark_tv_1d\u003e`_\n     -\n     - |Build Status TV1D|\n   * - `2D Total Variation Denoising \u003chttps://github.com/benchopt/benchmark_tv_2d\u003e`_\n     -\n     - |Build Status TV2D|\n   * - `ResNet Classification \u003chttps://github.com/benchopt/benchmark_resnet_classif\u003e`_\n     - `Results \u003chttps://benchopt.github.io/results/benchmark_resnet_classif.html\u003e`__\n     - |Build Status ResNetClassif|\n   * - `Bilevel Optimization \u003chttps://github.com/benchopt/benchmark_bilevel\u003e`_\n     - `Results \u003chttps://benchopt.github.io/results/benchmark_bilevel.html\u003e`__\n     - |Build Status Bilevel|\n\n\n\n\n.. |Test Status| image:: https://github.com/benchopt/benchopt/actions/workflows/test.yml/badge.svg\n   :target: https://github.com/benchopt/benchopt/actions/workflows/test.yml\n.. |Python 3.6+| image:: https://img.shields.io/badge/python-3.6%2B-blue\n   :target: https://www.python.org/downloads/release/python-360/\n.. |Documentation| image:: https://img.shields.io/badge/documentation-latest-blue\n   :target: https://benchopt.github.io\n.. |codecov| image:: https://codecov.io/gh/benchopt/benchopt/branch/main/graph/badge.svg\n   :target: https://codecov.io/gh/benchopt/benchopt\n.. |SWH| image:: https://archive.softwareheritage.org/badge/origin/https://github.com/benchopt/benchopt/\n    :target: https://archive.softwareheritage.org/browse/origin/?origin_url=https://github.com/benchopt/benchopt\n.. |discord| image:: https://dcbadge.vercel.app/api/server/EA2HGQb7nv?style=flat\n   :target: https://discord.gg/EA2HGQb7nv\n.. |install-per-months| image:: https://static.pepy.tech/badge/benchopt/month\n   :target: https://pepy.tech/project/benchopt\n\n.. |Build Status OLS| image:: https://github.com/benchopt/benchmark_ols/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_ols/actions\n.. |Build Status NNLS| image:: https://github.com/benchopt/benchmark_nnls/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_nnls/actions\n.. |Build Status Lasso| image:: https://github.com/benchopt/benchmark_lasso/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_lasso/actions\n.. |Build Status Lasso Path| image:: https://github.com/jolars/benchmark_lasso_path/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_lasso_path/actions\n.. |Build Status ElasticNet| image:: https://github.com/benchopt/benchmark_elastic_net/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_elastic_net/actions\n.. |Build Status MCP| image:: https://github.com/benchopt/benchmark_mcp/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_mcp/actions\n.. |Build Status LogRegL2| image:: https://github.com/benchopt/benchmark_logreg_l2/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_logreg_l2/actions\n.. |Build Status LogRegL1| image:: https://github.com/benchopt/benchmark_logreg_l1/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_logreg_l1/actions\n.. |Build Status HuberL2| image:: https://github.com/benchopt/benchmark_huber_l2/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_huber_l2/actions\n.. |Build Status QuantileRegL1| image:: https://github.com/benchopt/benchmark_quantile_regression/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_quantile_regression/actions\n.. |Build Status LinearSVM| image:: https://github.com/benchopt/benchmark_linear_svm_binary_classif_no_intercept/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_linear_svm_binary_classif_no_intercept/actions\n.. |Build Status LinearICA| image:: https://github.com/benchopt/benchmark_linear_ica/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_linear_ica/actions\n.. |Build Status JointDiag| image:: https://github.com/benchopt/benchmark_jointdiag/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_jointdiag/actions\n.. |Build Status TV1D| image:: https://github.com/benchopt/benchmark_tv_1d/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_tv_1d/actions\n.. |Build Status TV2D| image:: https://github.com/benchopt/benchmark_tv_2d/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_tv_2d/actions\n.. |Build Status ResNetClassif| image:: https://github.com/benchopt/benchmark_resnet_classif/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_resnet_classif/actions\n.. |Build Status Bilevel| image:: https://github.com/benchopt/benchmark_bilevel/actions/workflows/main.yml/badge.svg\n   :target: https://github.com/benchopt/benchmark_bilevel/actions\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenchopt%2Fbenchopt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbenchopt%2Fbenchopt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenchopt%2Fbenchopt/lists"}