{"id":22301703,"url":"https://github.com/jhorzek/lesstimate","last_synced_at":"2026-05-16T06:36:27.469Z","repository":{"id":152517338,"uuid":"626283952","full_name":"jhorzek/lesstimate","owner":"jhorzek","description":"Optimize your models with lasso, scad, mcp, and other penalty functions","archived":false,"fork":false,"pushed_at":"2025-10-16T23:50:13.000Z","size":2885,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-18T02:46:28.738Z","etag":null,"topics":["lasso","mcp","optimization","optimization-algorithms","regularization","scad"],"latest_commit_sha":null,"homepage":"https://jhorzek.github.io/lesstimate/","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jhorzek.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-11T06:50:25.000Z","updated_at":"2025-10-16T23:49:54.000Z","dependencies_parsed_at":"2024-01-11T19:46:18.567Z","dependency_job_id":"5b727480-832a-435f-8f85-2eec5663e251","html_url":"https://github.com/jhorzek/lesstimate","commit_stats":null,"previous_names":["jhorzek/lesspar","jhorzek/lesstimate","jhorzek/lessoptimizers"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jhorzek/lesstimate","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhorzek%2Flesstimate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhorzek%2Flesstimate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhorzek%2Flesstimate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhorzek%2Flesstimate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jhorzek","download_url":"https://codeload.github.com/jhorzek/lesstimate/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhorzek%2Flesstimate/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33092718,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-16T04:41:52.686Z","status":"ssl_error","status_checked_at":"2026-05-16T04:41:52.009Z","response_time":115,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lasso","mcp","optimization","optimization-algorithms","regularization","scad"],"created_at":"2024-12-03T18:23:57.678Z","updated_at":"2026-05-16T06:36:27.453Z","avatar_url":"https://github.com/jhorzek.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# lesstimate \u003cimg src=\"lesstimate_logo.png\" align=\"right\" height=\"150\" /\u003e\n\n\u003e A longer documentation of the library can be found at https://jhorzek.github.io/lesstimate/.\n\n**lesstimate** (**l**esstimate **es**timates **s**parse es**timate**s) is a C++ header-only library that lets you combine statistical models such linear regression with state of the art penalty functions (e.g., lasso, elastic net, scad). With **lesstimate** you can add regularization and variable selection procedures to your existing modeling framework. It is currently used in [**lessSEM**](https://github.com/jhorzek/lessSEM) to regularize structural equation models.\n\n## Features \n\n- **Multiple penalty functions**: **lesstimate** lets you apply any of the following penalties: ridge, lasso, adaptive lasso, elastic net, cappedL1, lsp, scad, mcp. Furthermore, you can combine multiple penalties.\n- **State of the art optimizers**: **lesstimate** provides two state of the art optimizers--variants of glmnet and ista.\n- **Header-only**: **lesstimate** is designed as a header-only library. Include the headers and you are ready to go.\n- **Builds on armadillo**: **lesstimate** builds on the popular *C++* [**armadillo**](https://arma.sourceforge.net/docs.html) library, providing you with access to a wide range of mathematical functions to create your model.\n- **R and C++**: **lesstimate** can be used in both, [*R*](https://github.com/jhorzek/lesstimateTemplateR) and [*C++*](https://github.com/jhorzek/lesstimateTemplateCpp) libraries.\n\n## Details\n\n**lesstimate** lets you optimize fitting functions of the form\n\n$$g(\\pmb\\theta) = f(\\pmb\\theta) + p(\\pmb\\theta),$$\n\nwhere $f(\\pmb\\theta)$ is a smooth objective function (e.g., residual sum squared, weighted least squares or log-Likelihood) and $p(\\pmb\\theta)$ is a non-smooth penalty function (e.g., lasso or scad).\n\nTo use the optimziers, you will need two functions:\n\n1. a function that computes the fit value $f(\\pmb\\theta)$ of your model\n2. a functions that computes the gradients $\\triangledown_{\\pmb\\theta}f(\\pmb\\theta)$ of the model\n\nGiven these two functions, **lesstimate** lets you apply any of the aforementioned penalties with the quasi-Newton glmnet optimizer developed by Friedman et al. (2010) and Yuan et al. (2012) or variants of the proximal-operator based ista optimizer (see e.g., Gong et al., 2013). Because both optimziers provide a very similar interface, sitching between them is fairly simple. This interface is inspired by the [**ensmallen**](https://ensmallen.org/) library. \n\nA thorough introduction to **lesstimate** and its use in R or C++ can be found in the [documentation](https://jhorzek.github.io/lesstimate/). \nWe also provide a [template for using **lesstimate** in R](https://github.com/jhorzek/lesstimateTemplateR) and [template for using **lesstimate** in C++](https://github.com/jhorzek/lesstimateTemplateCpp). Finally, you will find another example for including **lesstimate** in R in the package [**lessLM**](https://github.com/jhorzek/lessLM). We recommend that you use the [simplified interfaces](https://github.com/jhorzek/lesstimate/blob/main/include/simplified_interfaces.h) to get started. \n\n## Example\n\nThe following code demonstrates the use of **lesstimate** with regularized linear regressions. A longer step-by-step introduction including installation of **lesstimate** is provided in the [documentation](https://jhorzek.github.io/lesstimate/). The cmake infrastructure was created with [cmake-init](https://github.com/friendlyanon/cmake-init).\n\n```\n#include \u003carmadillo\u003e\n#include \u003clesstimate.h\u003e\n\n// The model must inherit from less::model and override the fit and (optionally) the gradients function.\nclass linearRegressionModel : public less::model\n{\npublic:\n    double fit(arma::rowvec b, less::stringVector labels) override\n    {\n        // compute sum of squared errors\n        arma::mat sse = (arma::trans(y - X * b.t()) * (y - X * b.t())) / (2.0 * y.n_elem);\n        return (sse(0, 0));\n    }\n\n    // linear regression requires dependent variables y and design matrix X:\n    const arma::colvec y;\n    const arma::mat X;\n\n    // constructor\n    linearRegressionModel(arma::colvec y_, arma::mat X_) : y(y_), X(X_){};\n};\n\nint main()\n{\n    // examples for design matrix and dependent variable:\n    arma::mat X = {{1.00, -0.70, -0.86},\n                   {1.00, -1.20, -2.10},\n                   {1.00, -0.15, 1.13},\n                   {1.00, -0.50, -1.50},\n                   {1.00, 0.83, 0.44},\n                   {1.00, -1.52, -0.72},\n                   {1.00, 1.40, -1.30},\n                   {1.00, -0.60, -0.59},\n                   {1.00, -1.10, 2.00},\n                   {1.00, -0.96, -0.20}};\n\n    arma::colvec y = {{0.56},\n                      {-0.32},\n                      {0.01},\n                      {-0.09},\n                      {0.18},\n                      {-0.11},\n                      {0.62},\n                      {0.72},\n                      {0.52},\n                      {0.12}};\n\n    // initialize model\n    linearRegressionModel linReg(y, X);\n\n    // To use the optimizers, you will need to\n    // (1) specify starting values and names for the parameters\n    arma::rowvec startingValues(3);\n    startingValues.fill(0.0);\n    std::vector\u003cstd::string\u003e labels{\"b0\", \"b1\", \"b2\"};\n    less::stringVector parameterLabels(labels);\n\n    // (2) specify the penalty to be used for each of the parameters:\n    std::vector\u003cstd::string\u003e penalty{\"none\", \"lasso\", \"lasso\"};\n\n    // (3) specify the tuning parameters of your penalty for\n    // each parameter:\n    arma::rowvec lambda = {{0.0, 0.2, 0.2}};\n    // theta is not used by the lasso penalty:\n    arma::rowvec theta = {{0.0, 0.0, 0.0}};\n\n    less::fitResults fitResult_ = less::fitGlmnet(\n        linReg,\n        startingValues,\n        parameterLabels,\n        penalty,\n        lambda,\n        theta);\n        \n    return(0);\n}\n```\n\n# References\n\n## Penalty Functions\n\n* Candès, E. J., Wakin, M. B., \u0026 Boyd, S. P. (2008). Enhancing Sparsity by \nReweighted l1 Minimization. Journal of Fourier Analysis and Applications, 14(5–6), \n877–905. https://doi.org/10.1007/s00041-008-9045-x\n* Fan, J., \u0026 Li, R. (2001). Variable selection via nonconcave penalized \nlikelihood and its oracle properties. Journal of the American Statistical \nAssociation, 96(456), 1348–1360. https://doi.org/10.1198/016214501753382273\n* Hoerl, A. E., \u0026 Kennard, R. W. (1970). Ridge Regression: Biased Estimation \nfor Nonorthogonal Problems. Technometrics, 12(1), 55–67. https://doi.org/10.1080/00401706.1970.10488634\n* Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. \nJournal of the Royal Statistical Society. Series B (Methodological), 58(1), 267–288.\n* Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. \nThe Annals of Statistics, 38(2), 894–942. https://doi.org/10.1214/09-AOS729\n* Zhang, T. (2010). Analysis of Multi-stage Convex Relaxation for Sparse Regularization. \nJournal of Machine Learning Research, 11, 1081–1107.\n* Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the \nAmerican Statistical Association, 101(476), 1418–1429. https://doi.org/10.1198/016214506000000735\n* Zou, H., \u0026 Hastie, T. (2005). Regularization and variable selection via the \nelastic net. Journal of the Royal Statistical Society: Series B, 67(2), 301–320. \nhttps://doi.org/10.1111/j.1467-9868.2005.00503.x\n\n## Optimizer\n\n### GLMNET \n\n* Friedman, J., Hastie, T., \u0026 Tibshirani, R. (2010). Regularization paths for \ngeneralized linear models via coordinate descent. Journal of Statistical \nSoftware, 33(1), 1–20. https://doi.org/10.18637/jss.v033.i01\n* Yuan, G.-X., Ho, C.-H., \u0026 Lin, C.-J. (2012). An improved GLMNET for \nl1-regularized logistic regression. The Journal of Machine Learning Research, \n13, 1999–2030. https://doi.org/10.1145/2020408.2020421\n\n### Variants of ISTA\n\n* Beck, A., \u0026 Teboulle, M. (2009). A Fast Iterative Shrinkage-Thresholding \nAlgorithm for Linear Inverse Problems. SIAM Journal on Imaging Sciences, 2(1), \n183–202. https://doi.org/10.1137/080716542\n* Gong, P., Zhang, C., Lu, Z., Huang, J., \u0026 Ye, J. (2013). A general iterative \nshrinkage and thresholding algorithm for non-convex regularized optimization problems. \nProceedings of the 30th International Conference on Machine Learning, 28(2)(2), 37–45.\n* Parikh, N., \u0026 Boyd, S. (2013). Proximal Algorithms. Foundations and \nTrends in Optimization, 1(3), 123–231.\n\n# License\n\nThis project is under GPL \u003e= 2.\n\nTHE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, \nINCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, \nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE \nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, \nWHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN \nCONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjhorzek%2Flesstimate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjhorzek%2Flesstimate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjhorzek%2Flesstimate/lists"}