{"id":20290021,"url":"https://github.com/abess-team/skscope","last_synced_at":"2025-05-15T08:10:47.580Z","repository":{"id":103983496,"uuid":"575404625","full_name":"abess-team/skscope","owner":"abess-team","description":"skscope: Sparse-Constrained OPtimization via itErative-solvers","archived":false,"fork":false,"pushed_at":"2024-10-26T05:59:02.000Z","size":24372,"stargazers_count":331,"open_issues_count":7,"forks_count":15,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-06T21:49:44.644Z","etag":null,"topics":["auto-differentiation","jax","non-convex-optimization","nonlinear-optimization","python","scikit-learn","sparsity-optimization"],"latest_commit_sha":null,"homepage":"https://skscope.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/abess-team.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-07T12:45:43.000Z","updated_at":"2025-03-27T09:26:39.000Z","dependencies_parsed_at":"2023-10-15T03:35:59.516Z","dependency_job_id":"e54bf03e-b7dd-4ade-b9a5-115526298920","html_url":"https://github.com/abess-team/skscope","commit_stats":{"total_commits":455,"total_committers":11,"mean_commits":41.36363636363637,"dds":0.6175824175824176,"last_synced_commit":"fe44eb3fd535ee345fe968f2116d0948bb60d80b"},"previous_names":["abess-team/skscope"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abess-team%2Fskscope","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abess-team%2Fskscope/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abess-team%2Fskscope/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abess-team%2Fskscope/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/abess-team","download_url":"https://codeload.github.com/abess-team/skscope/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248890027,"owners_count":21178347,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["auto-differentiation","jax","non-convex-optimization","nonlinear-optimization","python","scikit-learn","sparsity-optimization"],"created_at":"2024-11-14T15:05:57.948Z","updated_at":"2025-04-14T13:42:09.602Z","avatar_url":"https://github.com/abess-team.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![](docs/source/_static/skscope.png)\n\n# ``skscope``: Fast Sparse-Constraint Optimization\n\n[![pypi](https://img.shields.io/pypi/v/skscope?logo=Pypi)](https://pypi.org/project/skscope)\n[![Conda version](https://img.shields.io/conda/vn/conda-forge/skscope.svg?logo=condaforge)](https://anaconda.org/conda-forge/skscope)\n[![Build](https://github.com/abess-team/skscope/actions/workflows/CI.yml/badge.svg)](https://github.com/abess-team/skscope/actions/workflows/CI.yml/badge.svg)\n[![codecov](https://codecov.io/gh/abess-team/skscope/branch/master/graph/badge.svg?token=XRD5BDMR2E)](https://codecov.io/gh/abess-team/skscope)\n[![docs](https://readthedocs.org/projects/skscope/badge/?version=latest)](https://skscope.readthedocs.io/en/latest/?badge=latest)\n[![pyversions](https://img.shields.io/pypi/pyversions/skscope)](https://img.shields.io/pypi/pyversions/skscope)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\n\u003c!-- [![Downloads](https://static.pepy.tech/badge/skscope)](https://pepy.tech/project/skscope) --\u003e\n\n## What is `skscope`?\n\n``skscope`` aims to make sparsity-constrained optimization (SCO) accessible to **everyone** because SCO holds immense potential across various domains, including machine learning, statistics, and signal processing. By providing a user-friendly interface, ``skscope`` empowers individuals from diverse backgrounds to harness the power of SCO and unlock its broad range of applications (see examples exhibited below).\n\n![](docs/source/first_page.png)\n\n## Installation\n\nThe recommended option for most users:\n  \n```bash\npip install skscope\n```\n\nFor Linux or Mac users, an alternative is\n\n```bash\nconda install skscope\n```\n\nIf you want to work with the latest development version, the further [installation instructions](https://skscope.readthedocs.io/en/latest/userguide/index.html) help you install from source.\n\n## Quick examples\n\nHere's a quick example showcasing how you can use three simple steps to perform feature selection via the ``skscope``:\n\n```python\nfrom skscope import ScopeSolver\nfrom sklearn.datasets import make_regression\nimport jax.numpy as jnp\n\n## generate data\nx, y, coef = make_regression(n_features=10, n_informative=3, coef=True)\n\n## 1. define loss function\ndef ols_loss(para):\n    return jnp.sum(jnp.square(y - x @ para))\n\n## 2. initialize the solver where 10 parameters in total and three of which are sparse\nsolver = ScopeSolver(10, 3) \n\n## 3. use the solver to optimized the objective\nparams = solver.solve(ols_loss) \n```\n\nBelow's another example illustrates that you can modify the objective function to address another totally different problem. \n\n```python\nimport numpy as np\nimport jax.numpy as jnp\nimport matplotlib.pyplot as plt\nfrom skscope import ScopeSolver\n\n## generate data\nnp.random.seed(2023)\nx = np.cumsum(np.random.randn(500)) # random walk with normal increment\n\n## 1. define loss function\ndef tf_objective(params):\n    return jnp.sum(jnp.square(x - jnp.cumsum(params)))  \n\n## 2. initialize the solver where 10 parameters in total and three of which are sparse\nsolver = ScopeSolver(len(x), 10)\n\n## 3. use the solver to optimized the objective\nparams = solver.solve(tf_objective)\n\ntf_x = jnp.cumsum(params)\nplt.plot(x, label='observation', linewidth=0.8)\nplt.plot(tf_x, label='filtering trend')\nplt.legend(); plt.show()\n```\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"docs/source/userguide/figure/tf.png\" width=\"300\"/\u003e\n\u003c/p\u003e\n\nThe above Figure shows that the solution of ``ScopeSolver`` now captures the main trend of the observed random work. Again, 4 lines of code help us attain the solution. \n\n## Example gallery\n\nSince ``skscope`` can easily be applied to diverse objective functions, we can definitely leverage it to develop various machine learning methods that is driven by SCO. In our example gallery, we supply 25 comprehensive statistical/machine learning examples to illustrate the versatility of ``skscope``. \n\n## Why ``skscope`` is versatile?\n\nThe high versatility of ``skscope`` in effectively addressing SCO problems are derived from two key factors: theoretical concepts and computational implementation. In terms of theoretical concepts, there have been remarkable advancements in SCO in recent years, offering a range of efficient iterative methods for solving SCO. Some of these algorithms exhibit elegance by only relying on the current parameters and gradients for the iteration process. On the other hand, significant progress has been made in automatic differentiation, a fundamental component of deep learning algorithms that plays a vital role in computing gradients. By ingeniously combining these two important advancements, ``skscope`` emerges as the pioneering tool capable of handling diverse sparse optimization tasks.\n\nWith ``skscope``, the creation of new machine learning methods becomes effortless, leading to the advancement of the \"sparsity idea\" in machine learning. This, in turn, facilitates the availability of a broader spectrum of machine learning algorithms for tackling real-world problems.\n\n## Software features\n\n- Support multiple state-of-the-art SCO solvers. Now, ``skscope`` has supported these algorithms: SCOPE, HTP, Grasp, IHT, OMP, and FoBa. \n\n- User-friendly API\n  \n  - zero-knowledge of SCO solvers: the state-of-the-art solvers in ``skscope`` has intuitive and highly unified APIs. \n  \n  - extensive documentation: ``skscope`` is fully documented and accompanied by example gallery and reproduction scripts.\n\n- Solving SCO and its generalization: \n  \n  - SCO: $\\arg\\min\\limits_{\\theta \\in R^p} f(\\theta) \\text{ s.t. } ||\\theta||_0 \\leq s$; \n  \n  - SCO for group-structure parameters: $\\arg\\min\\limits_{\\theta \\in R^p} f(\\theta) \\text{ s.t. } I(||\\theta_{G_i}||_2 \\neq 0) \\leq s$ where ${G_i}_{i=1}^q$ is a non-overlapping partition for $\\{1, \\ldots, p\\}$;\n  \n  - SCO when pre-selecting parameters in set $\\mathcal{P}$: $\\arg\\min\\limits_{\\theta \\in R^p} f(\\theta) \\text{ s.t. } ||\\theta_{\\mathcal{P}^c}||_0 \\leq s$. \n\n- Data science toolkit\n  \n  - Information criterion and cross-validation for selecting $s$\n  \n  - Portable interface for developing new machine-learning methods\n\n- Just-in-time-compilation compatibility\n\n## Benchmark\n\n- Support recovery accuracy\n\n| Methods      | Linear regression | Logistic regression | Trend filtering | Multi-task learning | Ising model | Nonlinear feature selection |\n| ------------ | ----------------- | ------------------- | --------------- | ------------------- | ----------- | --------------------------- |\n| `OMPSolver`   | 1.00(0.01)        | 0.91(0.05)          | 0.70(0.18)      | 1.00(0.00)          | 0.98(0.03)  | 0.77(0.09)                  |\n| `IHTSolver`   | 0.79(0.04)        | 0.97(0.03)          | 0.08(0.10)      | 0.97(0.02)          | 0.96(0.05)  | 0.78(0.09)                  |\n| `HTPSolver`   | 1.00(0.00)        | 0.84(0.05)          | 0.41(0.22)      | 1.00(0.00)          | 0.97(0.03)  | 0.78(0.09)                  |\n| `GraspSolver` | 1.00(0.00)        | 0.90(0.08)          | 0.58(0.23)      | 1.00(0.00)          | 0.99(0.01)  | 0.78(0.08)                  |\n| `FoBaSolver`  | 1.00(0.00)        | 0.92(0.06)          | 0.87(0.13)      | 1.00(0.00)          | 1.00(0.01)  | 0.77(0.09)                  |\n| `ScopeSolver` | 1.00(0.00)        | 0.94(0.04)          | 0.79(0.19)      | 1.00(0.00)          | 1.00(0.01)  | 0.77(0.09)                  |\n| `cvxpy`       | 0.83(0.17)        | 0.83(0.05)          | 0.19(0.22)      | 1.00(0.00)          | 0.94(0.04)  | 0.74(0.09)                  |\n\nAll solvers (except `IHTSolver`) in `skscope` consistently outperformed `cvxpy` in terms of accuracy for the selection of the support set. \n\n- Runtime (measured in seconds):\n\n| Methods      | Linear regression | Logistic regression | Trend filtering | Multi-task learning | Ising model  | Nonlinear feature selection |\n| ------------ | ----------------- | ------------------- | --------------- | ------------------- | ------------ | --------------------------- |\n| `OMPSolver`   | 0.62(0.11)        | 0.80(0.11)          | 0.03(0.00)      | 2.70(0.26)          | 1.39(0.13)   | 13.24(3.91)                 |\n| `IHTSolver`   | 0.23(0.05)        | 0.18(0.12)          | 0.30(0.06)      | 0.80(0.11)          | 0.98(0.08)   | 1.67(0.50)                  |\n| `HTPSolver`   | 0.50(0.14)        | 0.94(0.44)          | 0.03(0.01)      | 14.18(5.13)         | 3.41(1.22)   | 12.97(6.23)                 |\n| `GraspSolver` | 0.18(0.06)        | 2.55(0.86)          | 0.08(0.03)      | 0.54(0.28)          | 0.53(0.22)   | 3.06(0.75)                  |\n| `FoBaSolver`  | 3.71(0.50)        | 3.28(0.39)          | 0.13(0.02)      | 6.22(0.61)          | 11.10(1.04)  | 57.42(12.95)                |\n| `ScopeSolver` | 0.30(0.08)        | 1.20(2.14)          | 0.09(0.01)      | 1.14(0.89)          | 1.17(0.25)   | 7.78(2.23)                  |\n| `cvxpy`       | 14.59(5.60)       | 69.45(53.47)        | 0.47(0.16)      | 39.36(155.70)       | 32.26(17.88) | 534.49(337.72)              |\n\n`skscope` demonstrated significant computational advantages over `cvxpy`, exhibiting speedups ranging from approximately 3-500 times.\n\n## Software architecture\n\n![](docs/source/contribute/figure/architecture-scope.png)\n\n## Citation\n\nIf you use `skscope` or reference our tutorials in a presentation or publication, we would appreciate citations of our library.\n\nThe corresponding BibteX entry:\n\n```\n@article{JMLR:v25:23-1574,\n  author  = {Zezhi Wang and Junxian Zhu and Xueqin Wang and Jin Zhu and Huiyang Pen and Peng Chen and Anran Wang and Xiaoke Zhang},\n  title   = {skscope: Fast Sparsity-Constrained Optimization in Python},\n  journal = {Journal of Machine Learning Research},\n  year    = {2024},\n  volume  = {25},\n  number  = {290},\n  pages   = {1--9},\n  url     = {http://jmlr.org/papers/v25/23-1574.html}\n}\n```\n\n## Contributions\n\n\u003cdiv align=\"center\"\u003e\n\u003c!-- \u003cdetails\u003e --\u003e\n\n\u003csummary\u003e👏 Thanks for the following support 👏\u003c/summary\u003e\n\n### Stargazers\n\n\n\u003cdiv align=\"center\"\u003e\n\n[![Stargazers repo roster for @abess-team/skscope](http://reporoster.com/stars/abess-team/skscope)](https://github.com/abess-team/skscope/stargazers)\n\n\n\n\u003c/div\u003e\n\n### Forkers\n\n\u003cdiv align=\"center\" \u003e\n\n[![Forkers repo roster for @abess-team/skscope](http://reporoster.com/forks/abess-team/skscope)](https://github.com/abess-team/skscope/network/members)\n\n\u003c/div\u003e\n\u003cbr/\u003e\u003c/details\u003e\u003cbr/\u003e\n\n\u003c/div\u003e\n\nAny kind of contribution to `skscope` would be highly appreciated! Please check the [contributor's guide](https://skscope.readthedocs.io/en/latest/contribute/index.html).\n\n- Bug report via [github issues](https://github.com/abess-team/skscope/issues)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabess-team%2Fskscope","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabess-team%2Fskscope","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabess-team%2Fskscope/lists"}