{"id":16391289,"url":"https://github.com/gjjvdburg/sparsestep","last_synced_at":"2026-06-18T22:31:58.283Z","repository":{"id":56935345,"uuid":"80015292","full_name":"GjjvdBurg/SparseStep","owner":"GjjvdBurg","description":"SparseStep: Approximating the Counting Norm for Sparse Regularization","archived":false,"fork":false,"pushed_at":"2021-01-12T16:31:17.000Z","size":88,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-06-11T17:08:26.959Z","etag":null,"topics":["feature-selection","lasso-variants","r","regularized-linear-regression","sparse-regression","sparse-regularization"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1701.06967","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GjjvdBurg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-25T13:08:02.000Z","updated_at":"2024-06-11T17:08:26.960Z","dependencies_parsed_at":"2022-08-21T05:50:12.157Z","dependency_job_id":null,"html_url":"https://github.com/GjjvdBurg/SparseStep","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GjjvdBurg%2FSparseStep","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GjjvdBurg%2FSparseStep/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GjjvdBurg%2FSparseStep/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GjjvdBurg%2FSparseStep/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GjjvdBurg","download_url":"https://codeload.github.com/GjjvdBurg/SparseStep/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240232929,"owners_count":19769096,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["feature-selection","lasso-variants","r","regularized-linear-regression","sparse-regression","sparse-regularization"],"created_at":"2024-10-11T04:45:38.375Z","updated_at":"2026-06-13T04:30:17.055Z","avatar_url":"https://github.com/GjjvdBurg.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SparseStep R Package\n\nSparseStep is an R package for sparse regularized regression and provides an \nalternative to methods such as best subset selection, elastic net, lasso, and \nlars. The SparseStep method is introduced in the following paper:\n\n[SparseStep: Approximating the Counting Norm for Sparse \nRegularization](https://arxiv.org/abs/1701.06967) by G.J.J. van den Burg, \nP.J.F. Groenen, and A. Alfons (*Arxiv preprint arXiv:1701.06967 [stat.ME]*, \n2017).\n\nThis R package can be easily installed by running \n``install.packages('sparsestep')`` in R. If you use the package in your work, \nplease cite the above reference using, for instance, the following BibTeX \nentry:\n\n```bibtex\n@article{vandenburg2017sparsestep,\n  title = {{SparseStep}: Approximating the Counting Norm for Sparse Regularization},\n  author = {{Van den Burg}, G. J. J. and Groenen, P. J. F. and Alfons, A.},\n  journal = {arXiv preprint arXiv:1701.06967},\n  year = {2017}\n}\n```\n\n## Introduction\n\nThe SparseStep method solves the regression problem regularized with the \n[`l_0` norm](https://en.wikipedia.org/wiki/Lp_space#When_p_=_0). Since the \n`l_0` term is highly non-convex and therefore difficult to optimize, this \nnon-convexity is introduced gradually in SparseStep during optimization. As in \nother regularized regression methods such as ridge regression and lasso, a \nregularization parameter ``lambda`` can be specified to control the amount of \nregularization.  The choice of regularization parameter affects how many \nnon-zero variables remain in the final model.\n\nWe will give a quick guide to SparseStep using the Prostate dataset from the \nbook [Elements of Statistical \nLearning](https://web.stanford.edu/~hastie/ElemStatLearn/). \n\nWe will show a few examples of running SparseStep on the Prostate dataset from \nthe [lasso2](https://cran.r-project.org/web/packages/lasso2/index.html) \npackage. First we load the data and create a data matrix and outcome vector:\n\n```r\n\u003e prostate \u003c- read.table(\"http://statweb.stanford.edu/~tibs/ElemStatLearn/datasets/prostate.data\")\n\u003e X \u003c- prostate[prostate$train == T, c(-1, -10)]\n\u003e X \u003c- as.matrix(X)\n\u003e y \u003c- prostate[prostate$train == T, 1]\n\u003e y \u003c- as.vector(y)\n```\n\nThe easiest way to fit a SparseStep model is to use the ``path.sparsestep`` \nfunction. This estimates the entire path of solutions for the SparseStep model \nfor different values of the regularization parameter using a [golden section \nsearch](https://en.wikipedia.org/wiki/Golden-section_search) algorithm.\n\n```r\n\u003e path \u003c- path.sparsestep(X, y)\nFound maximum value of lambda: 2^( 7 )\nFound minimum value of lambda: 2^( -3 )\nRunning search in interval [ -3 , 7 ] ...\nRunning search in interval [ -3 , 2 ] ...\nRunning search in interval [ -3 , -0.5 ] ...\nRunning search in interval [ -3 , -1.75 ] ...\nRunning search in interval [ -0.5 , 2 ] ...\nRunning search in interval [ -0.5 , 0.75 ] ...\nRunning search in interval [ 0.125 , 0.75 ] ...\nRunning search in interval [ 2 , 7 ] ...\n\n\u003e plot(path, col=1:nrow(path$beta))     # col specifies colors to matplot\n\u003e legend('topleft', legend=rownames(path$beta), lty=1, col=1:nrow(path$beta))\n```\n\nIn the resulting plot we can see the coefficients of the features that are \nincluded in the model at different values of ``lambda``:\n\n![SparseStep regression on Prostate dataset](./.github/images/sparsestep_prostate_1.png)\n\nThe coefficients of the model can be obtained using ``coef(path)``, which \nreturns a sparse matrix:\n\n```r\n\u003e coef(path)\n9 x 9 sparse Matrix of class \"dgCMatrix\"\n                   s0           s1           s2          s3           s4         s5        s6       s7\nIntercept  1.31349155  1.313491553  1.313491553  1.31349155  1.313491553 1.31349155 1.3134916 1.313492\nlweight   -0.11336968 -0.113485291  .            .           .           .          .         .\nage        0.02010188  0.020182049  0.018605327  0.01491472  0.018704172 0.01623212 .         .\nlbph      -0.05698125 -0.059026246 -0.069116923  .           .           .          .         .\nsvi        0.03511645  .            .            .           .           .          .         .\nlcp        0.41845469  0.423398063  0.420516410  0.43806447  0.433449263 0.38174743 0.3887863 .\ngleason    0.22438690  0.222333394  0.236944796  0.23503609  .           .          .         .\npgg45     -0.00911273 -0.009084031 -0.008949463 -0.00853420 -0.004328518 .          .         .\nlpsa       0.57545508  0.580111724  0.561063637  0.53017309  0.528953966 0.51473225 0.5336907 0.754266\n                s8\nIntercept 1.313492\nlweight   .\nage       .\nlbph      .\nsvi       .\nlcp       .\ngleason   .\npgg45     .\nlpsa      .\n```\n\nNote that the final model included in ``coef(beta)`` is a intercept-only \nmodel, which is generally not very useful. Predicting out-of-sample data can \nbe done easily using the ``predict`` function.\n\nBy default SparseStep centers the regressors and outcome variable ``y`` and \nnormalizes the regressors ``X`` to ensure that the regularization is applied \nevenly among them and the intercept is not penalized. If you prefer to use a \nconstant term in the regression and penalize this as well, you'll have to \ntransform the input data and disable the intercept:\n\n```r\n\u003e Z \u003c- cbind(constant=1, X)\n\u003e path \u003c- path.sparsestep(Z, y, intercept=F)\n...\n\u003e plot(path, col=1:nrow(path$beta))\n\u003e legend('bottomright', legend=rownames(path$beta), lty=1, col=1:nrow(path$beta))\n```\n\nNote that since we add the constant through the data matrix it is subject to \nregularization and therefore sparsity:\n\n![SparseStep regression on Prostate dataset (with constant)](./.github/images/sparsestep_prostate_2.png)\n\nFor more information and examples, please see the documentation included with \nthe package. In particular, the following pages are good places to start:\n\n```r\n\u003e ?'sparsestep-package'\n\u003e ?sparsestep\n\u003e ?path.sparsestep\n```\n\n## Reference\n\nIf you use SparseStep in any of your projects, please cite the paper using the \ninformation available through the R command:\n\n    citation('sparsestep')\n\nor use the following BibTeX code:\n\n    @article{van2017sparsestep,\n      title = {{SparseStep}: Approximating the Counting Norm for Sparse Regularization},\n      author = {Gerrit J.J. {van den Burg} and Patrick J.F. Groenen and Andreas Alfons},\n      journal = {arXiv preprint arXiv:1701.06967},\n      archiveprefix = {arXiv},\n      year = {2017},\n      eprint = {1701.06967},\n      url = {https://arxiv.org/abs/1701.06967},\n      primaryclass = {stat.ME},\n      keywords = {Statistics - Methodology, 62J05, 62J07},\n    }\n\n## Notes\n\nThis package is licensed under GPLv3. Please see the LICENSE file for more \ninformation. If you have any questions or comments about this package, please \nopen an issue [on GitHub](https://github.com/GjjvdBurg/sparsestep) (don't \nhesitate, you're helping to make this project better for everyone!). If you \nprefer to use email, please write to ``gertjanvandenburg at gmail dot com``.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgjjvdburg%2Fsparsestep","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgjjvdburg%2Fsparsestep","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgjjvdburg%2Fsparsestep/lists"}