{"id":29253677,"url":"https://github.com/dmitribogdanov/mathematica-optimization-methods","last_synced_at":"2025-07-04T02:07:52.502Z","repository":{"id":165663093,"uuid":"558597541","full_name":"DmitriBogdanov/mathematica-optimization-methods","owner":"DmitriBogdanov","description":"Reference implementations of 22 optimization methods from 8 different families in Wolfram Mathematica","archived":false,"fork":false,"pushed_at":"2025-05-10T13:47:15.000Z","size":3100,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-10T14:36:47.125Z","etag":null,"topics":["barrier-method","conjugate-gradient-descent","gradient-descent","newton-method","optimization","optimization-algorithms","quasi-newton-method","simplex-method"],"latest_commit_sha":null,"homepage":"","language":"Mathematica","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DmitriBogdanov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-10-27T21:57:31.000Z","updated_at":"2025-05-10T13:47:19.000Z","dependencies_parsed_at":"2023-10-05T10:30:51.674Z","dependency_job_id":null,"html_url":"https://github.com/DmitriBogdanov/mathematica-optimization-methods","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/DmitriBogdanov/mathematica-optimization-methods","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DmitriBogdanov%2Fmathematica-optimization-methods","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DmitriBogdanov%2Fmathematica-optimization-methods/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DmitriBogdanov%2Fmathematica-optimization-methods/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DmitriBogdanov%2Fmathematica-optimization-methods/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DmitriBogdanov","download_url":"https://codeload.github.com/DmitriBogdanov/mathematica-optimization-methods/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DmitriBogdanov%2Fmathematica-optimization-methods/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263432376,"owners_count":23465577,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["barrier-method","conjugate-gradient-descent","gradient-descent","newton-method","optimization","optimization-algorithms","quasi-newton-method","simplex-method"],"created_at":"2025-07-04T02:07:51.275Z","updated_at":"2025-07-04T02:07:52.481Z","avatar_url":"https://github.com/DmitriBogdanov.png","language":"Mathematica","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Optimization methods collection\n\nContains various minimization methods in 8 Wolfram Mathematica packages:\n\n* Package 1 (1-dimensional dichotomy)\n  * Dichotomy method\n  * Golden-section search\n\n* Package 2 (gradient descent methods)\n  * Steepest descent\n  * Gradient descent with geometric step reduction\n\n* Package 3 (nonlinear conjugate gradient methods)\n  * Nonlinear conjugate gradient method\n  * Fletcher-Reeves method\n  * Polak-Ribière method\n  * Nonlinear conjugate gradient method with Hessian matrix\n\n* Package 4 (Newton's methods)\n  * Newton's method\n  * Modified Newton's method with step reduction\n\n* Package 5 (Quasi-Newton methods)\n  * Davidon-Fletcher-Powell method\n  * Broyden-Fletcher-Goldfarb-Shanno method\n  * Powell method\n  * McCormick method\n\n* Package 6 (direct search methods)\n  * Cyclic coordinate descent\n  * Pattern search (Hooke-Jeeves method)\n  * Rosenbrock method\n  * Powell method\n\n* Package 7 (simplex methods)\n  * Regular simplex method\n  * Downhill simplex method (Nelder-Mead method)\n\n* Package 8 (barrier methods)\n  * Interior penalty function method\n  * Exterior penalty function method\n\n## Package 1 (1-dimensional dichotomy) examples\n\nMinimization of a single argument function on a given interval. Both methods have exponential convergence.\n\n\u003cimg src=\"images/package1_functionPlot.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package1_dichotomy.png\" width=40% height=40%\u003e\n\n## Package 2 (gradient descent methods) examples\n\n* Requires gradient to be present and computable\n\nMinimization of a 2-argument function from a given initial point. Rosenbrock function is used for testing, minimization process can be dynamically tracked. All consequent plots support dynamics. Following example showcases steepest descent method.\n\n\u003cimg src=\"images/package2_functionPlot.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package2_residual.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package2_contours.png\" width=40% height=40%\u003e\n\n## Package 3 (nonlinear conjugate gradient methods) examples\n\n* Requires gradient to be present and computable\n* Conjugate gradient methods can follow narrow (ill-conditioned) valleys, where the steepest descent method slows down and follows a criss-cross pattern\n\nMinimization of a 2-argument function from a given initial point. Rosenbrock function is used for testing, minimization process can be dynamically tracked. Following example showcases nonlinear conjugate gradient method with Hessian matrix.\n\n\u003cimg src=\"images/package3_functionPlot.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package3_residual.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package3_contours.png\" width=40% height=40%\u003e\n\n\n## Package 4 (Newton's methods) examples\n\n* Requires Hessian to be present and computable\n* Newton's method can minimize any polynomial of Nth order in N-1 steps\n\nMinimization of a 2-argument function from a given initial point. Rosenbrock function is used for testing, minimization process can be dynamically tracked. Following example showcases modified Newton's method with step reduction.\n\n\u003cimg src=\"images/package4_functionPlot.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package4_residual.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package4_contours.png\" width=40% height=40%\u003e\n\n## Package 5 (Quasi-Newton methods) examples\n\n* Quasi-Newton methods (except McCormick) keep most properties of a regular Newton's method without requiring Hessian computation\n\nMinimization of a 2-argument function from a given initial point. Rosenbrock function is used for testing, minimization process can be dynamically tracked. Following example showcases Powell method. \n\n\u003cimg src=\"images/package5_functionPlot.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package5_residual.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package5_contours.png\" width=40% height=40%\u003e\n\n## Package 6 (direct search methods) examples\n\n* Direct search methods do not require a gradient, thus can be used with non-continuous and non-differentiable functions\n\nMinimization of a 2-argument function from a given initial point. Rosenbrock function is used for testing, minimization process can be dynamically tracked. Following example showcases cyclic coordinate descent.\n\n\u003cimg src=\"images/package6_functionPlot.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package6_residual.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package6_contours.png\" width=40% height=40%\u003e\n\n## Package 7 (simplex methods) examples\n\n* Simplex methods do not require a gradient, thus can be used with non-continuous and non-differentiable functions\n\nMinimization of a 2-argument function from a given initial point. Rosenbrock function is used for testing, minimization process can be dynamically tracked. Following example showcases Nelder-Mead method.\n\n\u003cimg src=\"images/package7_functionPlot.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package7_residual.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package7_contours.png\" width=40% height=40%\u003e\n\n## Package 8 (barrier methods) examples\n\n* Barrier methods use penalty functions to restrict minimization to a given region g(x, y) \u003c 0\n* Through generally reliable, a large enough step can overshoot the barrier, which leads to a longer convergence\n\nMinimization of a 2-argument function from a given initial point restricted to a region. Rosenbrock function is used for testing. Following example showcases exterior barrier method.\n\n\u003cimg src=\"images/package8_functionPlot.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package8_exterior_contours1.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package8_exterior_contours2.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package8_exterior_penaltyPlot.png\" width=40% height=40%\u003e\n\nThe same problem with interior barriers:\n\n\u003cimg src=\"images/package8_interior_contours1.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package8_interior_contours2.png\" width=40% height=40%\u003e\n\n\u003cimg src=\"images/package8_interior_penaltyPlot.png\" width=40% height=40%\u003e\n\n## Usage\n\nConfig parameters at the top of each package to select function, method, precision, starting point, boundary and etc. Minimization can be tracked real-time and most consequent plots support dynamics.\n\n## Requirements\n\nTo launch Mathematica packages one may need a valid Wolfram Mathematica license.\n\nWolfram Engine can also be used for free to run packages without the Mathematica GUI.\n\nAs an alternative packages can be converted to Jypiter notebooks and executed with Wolfram Lang. \n\n## Version History\n\n* 01.02\n  * README fixes\n\n* 01.01\n  * Translated package 8, converted notebook to Mathematica package\n  * Final touches to docs and comments\n\n* 00.06\n  * Translated package 7, converted notebook to Mathematica package\n\n* 00.05\n  * Translated package 6, converted notebook to Mathematica package\n  * Improved method descriptions\n\n* 00.04\n  * Translated package 5, converted notebook to Mathematica package\n\n* 00.03\n  * Translated package 4, converted notebook to Mathematica package\n\n* 00.02\n  * Translated package 3, converted notebook to Mathematica package\n\n* 00.01\n  * Translated package 1, altered some plotting methods, converted notebook to Mathematica package\n  * Translated package 2, altered some plotting methods, converted notebook to Mathematica package\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE.md file for details\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmitribogdanov%2Fmathematica-optimization-methods","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdmitribogdanov%2Fmathematica-optimization-methods","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmitribogdanov%2Fmathematica-optimization-methods/lists"}