{"id":21601532,"url":"https://github.com/simple-robotics/proxqp_benchmark","last_synced_at":"2025-04-11T02:13:27.402Z","repository":{"id":65259000,"uuid":"525485309","full_name":"Simple-Robotics/proxqp_benchmark","owner":"Simple-Robotics","description":"QP Benchmarks for the ProxQP Solver against OSQP, GUROBI, MOSEK, quadprog and qpOASES ","archived":false,"fork":false,"pushed_at":"2023-04-11T09:38:41.000Z","size":74083,"stargazers_count":29,"open_issues_count":2,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-11T02:13:14.026Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Simple-Robotics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-08-16T17:46:52.000Z","updated_at":"2025-02-12T11:16:19.000Z","dependencies_parsed_at":"2023-01-16T00:30:23.958Z","dependency_job_id":null,"html_url":"https://github.com/Simple-Robotics/proxqp_benchmark","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Simple-Robotics%2Fproxqp_benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Simple-Robotics%2Fproxqp_benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Simple-Robotics%2Fproxqp_benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Simple-Robotics%2Fproxqp_benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Simple-Robotics","download_url":"https://codeload.github.com/Simple-Robotics/proxqp_benchmark/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248328163,"owners_count":21085261,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-24T19:09:47.185Z","updated_at":"2025-04-11T02:13:27.384Z","avatar_url":"https://github.com/Simple-Robotics.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Benchmark examples for the PROXQP solver\n\nThese are the scripts to compare the following Quadratic Program (QP) solvers\n\n-   PROXQP (with dense backend)\n-   OSQP\n-   GUROBI\n-   MOSEK\n-   qpOASES\n-   quadprog\n\nThe whole API comes from the one developped by the authors of [OSQP](https://github.com/osqp/osqp_benchmarks/). We thank them a lot for their rigorous approach.\n\nThe detailed description of these tests is available in [this paper](https://arxiv.org/pdf/1711.08013.pdf).\n\nTo run these scripts you need standard python libraries: `pandas`, `matplotlib`, `numpy`, `scipy`, `scikit-learn`, and `cvxpy`, which can be installed using for example conda:\n```bash\nconda install pandas matplotlib numpy scipy cvxpy scikit-learn\n``` \nYou need further as well to install the solvers `gurobi`, `mosek`, `OSQP`, `quadprog` and `qpOASES`. The first four can be installed for example using conda:\n```bash\nconda install -c gurobi gurobi\nconda install -c mosek mosek\nconda install -c conda-forge osqp\nconda install -c omnia quadprog\n```\nGurobi and Mosek require a license for working. Note that if you are an academic, they provide specific and temporary free licenses.\n\nFinally, for installing qpOASES, you should look at their [install file](https://github.com/coin-or/qpOASES/blob/master/INSTALL.txt) in their github repository.\n\nAll the scripts come with options (default to `False`)\n\n- `--parallel` for parallel execution across instances\n- `--verbose` for verbose solvers output (they can be slower than necessary while printing)\n- `--high_accuracy` for high accuracy `eps=1e-09` solver settings + optimality checks\n\nNote that the parallel option has not be tested for the paper benchmarks.\n\n## Benchmark problems\nThe problems are all randomly generated as described in the [ProxQP paper](https://www.roboticsproceedings.org/rss18/p040.pdf).\nProblem instances include\n\n-   Inequality and Equality Constrained QP,\n-   Pure Inequality Degenerate QP,\n-   Pure Inequality QP without strictly convex Hessian matrix.\n\nWe generate the problems using the scripts in the `problem_classes` folder.\n\nTo execute these tests run\n```python\npython run_benchmark_problems.py\n```\n\n### Results\n\nFor Convex Quadratic Programs with inequality and equality constraints, when asking for a high accuracy (e.g., 1.E-9), the resulting [shifted geometric means](http://plato.asu.edu/ftp/shgeom.html) are, for `dense` problems \n\n| PROXQP | quadprog          | OSQP            | GUROBI             | MOSEK              | qpOASES            |\n| -----  | ----------------- | --------------- | ------------------ | ------------------ | ------------------ |\n| 1.0    | 20.4              | 18.2            | 5177.8             | 38131.0            | 282.6              |\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"time_series_barplot_Random Mixed QP_dense_high_accuracy.jpg\" width=\"700\" alt=\"Random Mixed QP_dense_high_accuracy\" align=\"center\"/\u003e\n\u003c/p\u003e\n\nand for `sparser` one (i.e., a ratio of sparsity of around 15%)\n\n| PROXQP | quadprog          | OSQP            | GUROBI             | MOSEK              | qpOASES            |\n| -----  | ----------------- | --------------- | ------------------ | ------------------ | ------------------ |\n| 1.0    | 19.3              | 3.5             | 6938.0             | 76844.2            | 260.6              |\n\n\nThese results have been obtained with a 11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz CPU.\n\n## Maros Meszaros problems\nThese are the hard problems from the [Maros Meszaros testset](http://www.cuter.rl.ac.uk/Problems/marmes.shtml) converted using [CUTEst](https://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki) and the scripts in the [maros_meszaros_data/](./problem_classes/maros_meszaros_data) folder.\n\nTo execute these tests run\n```python\npython run_maros_meszaros_problems.py\n```\n\n### Results\nThe resulting [shifted geometric means](http://plato.asu.edu/ftp/shgeom.html) are when asking for a high accuracy (e.g., 1.E-9)\n\n| PROXQP | quadprog          | OSQP            | GUROBI             | MOSEK              | qpOASES            |\n| -----  | ----------------- | --------------- | ------------------ | ------------------ | ------------------ |\n| 1.0    | 105.8             | 14.2            | 114.2              | 194.9              | 4.5                |\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"performance_profile_maros_meszaros_problems_high_accuracy.jpg\" width=\"700\" alt=\"maros_meszaros_problems_high_accuracy\" align=\"center\"/\u003e\n\u003c/p\u003e\n\nAt lower accuracy (e.g., 1.E-3), the resulting shifted geometric means are\n\n| PROXQP | quadprog          | OSQP            | GUROBI             | MOSEK              | qpOASES            |\n| -----  | ----------------- | --------------- | ------------------ | ------------------ | ------------------ |\n| 1.0    | 1543.3            | 108.5           | 637.3              | 175.3              | 17.1               |\n\nThese results have been obtained as well with a 11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz CPU.\n\n## Citing\n\nIf you are using these benchmarks for your work, please cite the [ProxQP paper](https://www.roboticsproceedings.org/rss18/p040.pdf):\n```bash\n@inproceedings{bambade:hal-03683733,\n  TITLE = {{PROX-QP: Yet another Quadratic Programming Solver for Robotics and beyond}},\n  AUTHOR = {Antoine Bambade, Sarah El-Kazdadi, Adrien Taylor, Justin Carpentier},\n  URL = {https://hal.inria.fr/hal-03683733},\n  BOOKTITLE = {{RSS 2022 - Robotics: Science and Systems}},\n  ADDRESS = {New York, United States},\n  YEAR = {2022},\n  MONTH = June,\n  PDF = {https://hal.inria.fr/hal-03683733/file/Yet_another_QP_solver_for_robotics_and_beyond.pdf},\n  HAL_ID = {hal-03683733},\n  HAL_VERSION = {v1},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimple-robotics%2Fproxqp_benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimple-robotics%2Fproxqp_benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimple-robotics%2Fproxqp_benchmark/lists"}