{"id":31392903,"url":"https://github.com/aidinhamedi/optimizer-benchmark","last_synced_at":"2026-01-05T09:14:31.970Z","repository":{"id":309607945,"uuid":"1035905701","full_name":"AidinHamedi/Optimizer-Benchmark","owner":"AidinHamedi","description":"A benchmarking suite for evaluating PyTorch optimization algorithms on 2D mathematical functions (optimizer benchmark)","archived":false,"fork":false,"pushed_at":"2025-09-22T18:14:07.000Z","size":304003,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-22T20:24:13.420Z","etag":null,"topics":["benchmark","deep-learning","functions","machine-learning","math","mathematics","mathfunctions","ml","optimization","optimization-algorithms","optimizer","optimizer-visualization","python","python3","pytorch","test"],"latest_commit_sha":null,"homepage":"https://aidinhamedi.github.io/Optimizer-Benchmark/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AidinHamedi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-11T09:14:49.000Z","updated_at":"2025-09-22T18:14:12.000Z","dependencies_parsed_at":"2025-08-12T21:29:20.178Z","dependency_job_id":"2d966f3a-37a3-4d25-9663-61f04eb9a24f","html_url":"https://github.com/AidinHamedi/Optimizer-Benchmark","commit_stats":null,"previous_names":["aidinhamedi/ml-optimizer-benchmark","aidinhamedi/optimizer-benchmark"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/AidinHamedi/Optimizer-Benchmark","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AidinHamedi%2FOptimizer-Benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AidinHamedi%2FOptimizer-Benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AidinHamedi%2FOptimizer-Benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AidinHamedi%2FOptimizer-Benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AidinHamedi","download_url":"https://codeload.github.com/AidinHamedi/Optimizer-Benchmark/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AidinHamedi%2FOptimizer-Benchmark/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":277467483,"owners_count":25822917,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-29T02:00:09.175Z","response_time":84,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","deep-learning","functions","machine-learning","math","mathematics","mathfunctions","ml","optimization","optimization-algorithms","optimizer","optimizer-visualization","python","python3","pytorch","test"],"created_at":"2025-09-29T04:54:56.832Z","updated_at":"2026-01-05T09:14:31.963Z","avatar_url":"https://github.com/AidinHamedi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Optimizer Benchmark\n[![Deploy Benchmark Site](https://github.com/AidinHamedi/Optimizer-Benchmark/actions/workflows/deploy.yml/badge.svg)](https://github.com/AidinHamedi/Optimizer-Benchmark/actions/workflows/deploy.yml)\n\nA benchmarking suite for evaluating and comparing PyTorch optimization algorithms on 2D mathematical functions.\n\n## 🌟 Highlights\n\n*   Benchmarks optimizers from the `pytorch_optimizer` library.\n*   Uses Optuna for hyperparameter tuning.\n*   Generates trajectory visualizations for each optimizer and function.\n*   Presents performance rankings on a project website.\n*   Configurable via a `config.toml` file.\n\n## ℹ️ Overview\n\nThis project provides a framework to evaluate and compare the performance of various PyTorch optimizers. It uses algorithms from `pytorch_optimizer` and performs hyperparameter searches with Optuna. The benchmark is run on a suite of standard 2D mathematical test functions, and the results, including optimization trajectories, are visualized and ranked.\n\n\u003e [!WARNING]\n\u003e **Important Limitations**: These benchmark results are based on synthetic 2D functions and may not reflect real-world performance when training actual neural networks. The rankings should only be used as a reference, not as definitive guidance for choosing optimizers in practical applications.\n\n## 📌 Benchmark Functions\n\nThe optimizers are evaluated on the following standard 2D test functions. Click on a function's name to learn more about it.\n\n| Function                                                                                   | Function                                                                                             |\n| :----------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------- |\n| [Ackley](https://www.sfu.ca/~ssurjano/ackley.html)                                         | [Lévy N. 13](https://www.sfu.ca/~ssurjano/levy13.html)                                               |\n| [Langermann](https://www.sfu.ca/~ssurjano/langer.html)                                     | [Eggholder](https://www.sfu.ca/~ssurjano/egg.html)                                                   |\n| [Gramacy \u0026 Lee](https://www.sfu.ca/~ssurjano/grlee12.html)                                 | [Griewank](https://www.sfu.ca/~ssurjano/griewank.html)                                               |\n| [Rastrigin](https://www.sfu.ca/~ssurjano/rastr.html)                                       | [Rosenbrock](https://www.sfu.ca/~ssurjano/rosen.html)                                                |\n| [Weierstrass](https://en.wikipedia.org/wiki/Weierstrass_function)                          | [Styblinski–Tang](https://www.sfu.ca/~ssurjano/stybtang.html)                                        |\n| [Goldstein-Price](https://www.sfu.ca/~ssurjano/goldpr.html)                                | [Gradient Labyrinth](https://aidinhamedi.github.io/Optimizer-Benchmark/functions/gradient_labyrinth) |\n| [Neural Canyon](https://aidinhamedi.github.io/Optimizer-Benchmark/functions/neural_canyon) | [Quantum Well](https://aidinhamedi.github.io/Optimizer-Benchmark/functions/quantum_well)             |\n| [Beale](https://www.sfu.ca/~ssurjano/beale.html)                                           |                                                                                        \n\n\n## 📊 Results \u0026 Visualizations\n\nThe full benchmark results, including performance rankings and detailed trajectory plots for each optimizer, are available on the project website.\n\n#### ➡️ [**View the Optimizer Benchmark Website (Rankings \u0026 Visualizations)**](https://aidinhamedi.github.io/Optimizer-Benchmark/)\n#### ➡️ [**Download the Benchmark Results**](https://github.com/Aidinhamedi/Optimizer-Benchmark/releases/latest)\n\n## 🚀 Quick Start\n\n```bash\n# Clone repository\ngit clone --depth 1 https://github.com/AidinHamedi/Optimizer-Benchmark.git\ncd Optimizer-Benchmark\n\n# Install dependencies\nuv sync\n\n# Run the benchmark\npython runner.py\n```\n\nThe script will load settings from `config.toml`, run hyperparameter tuning for each optimizer, and save the results and visualizations to the `./results/` directory.\n\n## 🤝 Contributing\n\nContributions are welcome! In particular, I’m looking for help improving and expanding the **web page**.\n\nIf you’d like to contribute, please feel free to submit a pull request or open an issue to discuss your ideas.\n\n## 📚 References\n\n*   Virtual Library of Simulation Experiments: *Test Functions and Datasets for Optimization Algorithms*.\n    Source: Simon Fraser University\n    [https://www.sfu.ca/~ssurjano/optimization.html](https://www.sfu.ca/~ssurjano/optimization.html)\n    Curated by Derek Bingham — For inquiries: dbingham@stat.sfu.ca\n\n*   Kim, H. (2021). *pytorch\\_optimizer: optimizer \u0026 lr scheduler \u0026 loss function collections in PyTorch* (Version 2.12.0) \\[Computer software].\n    [https://github.com/kozistr/pytorch\\_optimizer](https://github.com/kozistr/pytorch_optimizer)\n\n## 📝 License\n\n\u003cpre\u003e\n Copyright (c) 2025 Aidin Hamedi\n\n This software is released under the MIT License.\n https://opensource.org/licenses/MIT\n\u003c/pre\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faidinhamedi%2Foptimizer-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faidinhamedi%2Foptimizer-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faidinhamedi%2Foptimizer-benchmark/lists"}