{"id":16508705,"url":"https://github.com/mobeets/group-ard","last_synced_at":"2026-06-18T23:32:02.115Z","repository":{"id":208688696,"uuid":"722246731","full_name":"mobeets/group-ARD","owner":"mobeets","description":"code for performing Bayesian ARD regression, where covariates have groups","archived":false,"fork":false,"pushed_at":"2024-03-25T19:52:40.000Z","size":314,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-01T11:49:52.233Z","etag":null,"topics":["automatic-relevance-determination","bayesian-linear-regression","bayesian-regression","sparse-regression"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mobeets.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-22T18:48:58.000Z","updated_at":"2023-11-22T19:26:19.000Z","dependencies_parsed_at":"2024-11-13T09:04:50.608Z","dependency_job_id":"01c80bd2-df33-4330-af99-7ac44eb5bb1d","html_url":"https://github.com/mobeets/group-ARD","commit_stats":null,"previous_names":["mobeets/group-ard"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mobeets/group-ARD","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mobeets%2Fgroup-ARD","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mobeets%2Fgroup-ARD/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mobeets%2Fgroup-ARD/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mobeets%2Fgroup-ARD/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mobeets","download_url":"https://codeload.github.com/mobeets/group-ARD/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mobeets%2Fgroup-ARD/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34511617,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-18T02:00:06.871Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automatic-relevance-determination","bayesian-linear-regression","bayesian-regression","sparse-regression"],"created_at":"2024-10-11T15:47:06.386Z","updated_at":"2026-06-18T23:32:02.079Z","avatar_url":"https://github.com/mobeets.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Summary\n\nSuppose we have covariates $X \\in \\mathbb{R}^{N \\times K}$ and observations $y \\in \\mathbb{R}^N$, where our observation model is $y_i \\sim \\mathcal{N}(x_i^\\top w, \\sigma^2)$. Here, $w \\in \\mathbb{R}^K$ are unknown weights.\n\nIn linear regression we want to find the best estimate of the weights given $X$ and $y$. For example, standard linear regression finds the weights $\\widehat{w}$ minimizing the sum of the squared residuals:\n\n$$ \\|| y - X \\widehat{w} \\||_2^2 $$\n\nWhen $N$ is small or $K$ is large, it's often useful to do Bayesian linear regression. This involves choosing a prior on our weights (see [1] for more details). Some common choices or prior are:\n\n1. __Ridge__: $w_i \\sim \\mathcal{N}(0, \\alpha^{-1})$, where $\\alpha \\in \\mathbb{R}$ is called our \"inverse prior variance\".\n2. __Automatic Relevance Determination (ARD)__: $w_i \\sim \\mathcal{N}(0, \\alpha_i^{-1})$. Note that now each covariate has its own inverse prior variance.\n\nHere we consider a third option in between these two, which I will call \"Group ARD\" (in analogy to Group Lasso [2]). This prior is relevant when our covariates can be grouped. Specifically, we assume the $i^{th}$ covariate has a known group label $c_i \\in \\\\{ 1, 2, \\ldots, G\\\\}$, where $G$ is the total number of groups. The idea is that every covariate in the same group has the same inverse prior variance. In other words:\n\n3. __Group ARD__: $w \\sim \\mathcal{N}(0, \\alpha_{c_i}^{-1})$\n\nWe can estimate the inverse prior variances using similar methods proposed in Appendix 1 of [1].\n\nFor what it's worth, this model was implemented in [3] but called \"Group-sparse Bayesian linear discriminant analysis\" (?).\n\n## References\n\n[1] Tipping, Michael E. \"Sparse Bayesian learning and the relevance vector machine.\" Journal of machine learning research 1.Jun (2001): 211-244.\n\n[2] Yuan, Ming, and Yi Lin. \"Model selection and estimation in regression with grouped variables.\" Journal of the Royal Statistical Society Series B: Statistical Methodology 68.1 (2006): 49-67.\n\n[3] Yu, Tianyou, et al. \"Grouped automatic relevance determination and its application in channel selection for P300 BCIs.\" IEEE Transactions on Neural Systems and Rehabilitation Engineering 23.6 (2015): 1068-1077.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmobeets%2Fgroup-ard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmobeets%2Fgroup-ard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmobeets%2Fgroup-ard/lists"}