{"id":18758133,"url":"https://github.com/ankargren/bayesnorm","last_synced_at":"2026-06-08T16:03:29.582Z","repository":{"id":93923732,"uuid":"165235852","full_name":"ankargren/bayesnorm","owner":"ankargren","description":"Efficient sampling of normal posterior distributions","archived":false,"fork":false,"pushed_at":"2019-01-17T14:38:20.000Z","size":49609,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-13T18:51:04.001Z","etag":null,"topics":["cpp","package","r","rcpp","rcpparmadillo"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ankargren.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-11T11:51:33.000Z","updated_at":"2019-05-17T22:16:45.000Z","dependencies_parsed_at":"2023-08-28T00:15:22.134Z","dependency_job_id":null,"html_url":"https://github.com/ankargren/bayesnorm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ankargren/bayesnorm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankargren%2Fbayesnorm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankargren%2Fbayesnorm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankargren%2Fbayesnorm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankargren%2Fbayesnorm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ankargren","download_url":"https://codeload.github.com/ankargren/bayesnorm/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankargren%2Fbayesnorm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34069501,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-08T02:00:07.615Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","package","r","rcpp","rcpparmadillo"],"created_at":"2024-11-07T17:45:28.995Z","updated_at":"2026-06-08T16:03:29.564Z","avatar_url":"https://github.com/ankargren.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\noutput:\n  github_document\n---\n\n```{r setup, include=FALSE}\nknitr::opts_chunk$set(echo = TRUE)\n```\n\n# bayesnorm\n\n*Efficient sampling of normal posterior distributions*\n\n[![Build Status](https://travis-ci.org/ankargren/bayesnorm.svg?branch=master)](https://travis-ci.org/ankargren/bayesnorm)[![Coverage status](https://codecov.io/gh/ankargren/bayesnorm/branch/master/graph/badge.svg)](https://codecov.io/github/ankargren/bayesnorm?branch=master)\n\n## About\n\nThe `bayesnorm` package provides two functions, `rmvn_bcm` and `rmvn_rue`, which \nallow for efficient sampling from normal posterior distributions. The posterior distribution\nshould have the form `mu = Sigma * Phi' * alpha`, where the posterior\ncovariance matrix is `Sigma = (Phi' * Phi + D^{-1})^{-1}` and `D` is \na diagonal matrix. This is the ubiquitous form of normal posteriors. It shows up\nin standard Bayesian linear regression as well as when using scale mixtures such \nas the horseshoe prior or other priors from the global-local shrinkage family. The idea\nof this package is to provide C++ headers so that sampling routines specifically\ntailored for the form of the posterior can be employed, which can speed up computations\nnotably. The main use of the package is therefore for building and implementing\nGibbs samplers. In addition to the C++ headers, simple wrappers are available for\nuse from R directly.\n\n## Installation\n\nThe package is currently GitHub only and can be installed using `devtools`:\n```{r, eval = FALSE}\ndevtools::install_github(\"ankargren/bayesnorm\")\n```\n\n## Sampling routines\n\nThe `rmvn_bcm` is appropriate when `n\u003cp`, whereas `rmvn_rue` \nis typically the faster alternative when `n\u003ep`.\n\nThe sampling routines are based on the proposals by Bhattacharya, Chakraborty \nand Mallick (2016) and Rue (2001). The former is based on \nan idea of avoiding operations in the `p` dimension in favor of working\nin the `n` dimension, which is why it is preferrable when `n\u003cp`. The\nlatter sampling routine instead does the opposite. The Bhattacharya, Chakraborty \nand Mallick (2016) algorithm has complexity $O(n^2p)$, whereas the Rue (2001) routine\nhas complexity $O(p^3)$. \n\nThe C++ implementations are available as headers and can therefore be called\ndirectly in C++ (e.g. via Rcpp) if necessary by other packages.\n\n## Example 1: Mean of draws\n\nWe can first verify that the two functions draw from the correct distribution by\na simple comparison with the sampling function in the `mgcv` package. First generate\nsome toy data:\n```{r, cache = TRUE}\nlibrary(bayesnorm)\nset.seed(10871)\nX \u003c- matrix(rnorm(1000), 50, 20)\nd \u003c- runif(20, 0, 1)\nalpha \u003c- rnorm(50)\n```\n\nNext, call the two functions a large number of times:\n```{r, cache = TRUE}\nrue \u003c- replicate(100000, rmvn_rue(X, d, alpha), simplify = \"matrix\")\nbcm \u003c- replicate(100000, rmvn_bcm(X, d, alpha), simplify = \"matrix\")\n```\n\nLet's also contrast these results with using an approach where we compute the \nposterior mean and covariance and generate from a normal distribution with\nmean `mu` and covariance matrix `Sigma`. The results should be the same, but it\nwould be an inefficient way to do it.\n```{r, cache = TRUE}\nSigma \u003c- solve(crossprod(X) + diag(1/d))\nmu \u003c- Sigma %*% crossprod(X, alpha)\nmgcv \u003c- mgcv::rmvn(100000,c(mu),Sigma)\n```\n\nIf we now compute the mean over the draws, we can see that they are close:\n```{r}\ncbind(rue = rowMeans(rue),\n      bcm = rowMeans(bcm),\n      mgcv = colMeans(mgcv))\n```\nWhile this does not prove anything, it shows us that the means of a large number of\ndraws using the three functions are essentially the same.\n\n## Example 2: Efficiency of sampling routines\n\nFor the second example, we can investigate what use of these two functions mean in terms of efficiency. First, we load the `bayesnorm` and `tidyverse` packages:\n\n```{r}\nlibrary(bayesnorm)\nlibrary(tidyverse)\n```\n\nNext, set the seed and values of `n` and `p` to look at. The function `sample_fun()` computes the posterior mean and covariance and then makes a draw using the `mgcv::rmvn()` function.\n```{r}\nset.seed(16379)\nn_vec \u003c- c(50, 100, 200, 500)\np_vec \u003c- c(50, 100, 200, 500)\n\nsample_fun \u003c- function(X, d, alpha) {\n  Sigma \u003c- solve(crossprod(X) + diag(1/d))\n  mu \u003c- Sigma %*% crossprod(X, alpha)\n  out \u003c- mgcv::rmvn(1,c(mu),Sigma)\n  return(out)\n}\n```\n\nThe `data.frame` `combs` stores all the combinations of `n` and `p`; we will go through its rows and fill in timings for the three sampling functions:\n```{r}\ncombs \u003c- expand.grid(n = n_vec, p = p_vec, bcm = NA, mgcv = NA, rue = NA)\n```\n\nFinally, loop through `combs`, generate data and time the functions (NB: it takes some time):\n```{r, cache = TRUE}\nfor (i in 1:nrow(combs)) {\n  n \u003c- combs[i, 1]\n  p \u003c- combs[i, 2]\n  \n  X \u003c- matrix(rnorm(n*p), n, p)\n  d \u003c- runif(p, 0, 1)\n  alpha \u003c- rnorm(n)\n  mb \u003c- microbenchmark::microbenchmark(rue = rmvn_rue(X, d, alpha),\n                                 bcm = rmvn_bcm(X, d, alpha),\n                                 mgcv = sample_fun(X, d, alpha), times = 1000)\n  \n  combs[i, 3:5] \u003c- as_tibble(mb) %\u003e%\n    group_by(expr) %\u003e%\n    summarize(median = median(time)/1e6) %\u003e%\n    arrange(as.character(expr)) %\u003e% \n    pull(median)\n  \n}\n```\n\nBefore plotting, put the data into long format:\n```{r}\ntheme_set(theme_minimal())\ncbPalette \u003c- c(\"#999999\", \"#E69F00\", \"#56B4E9\", \"#009E73\", \n               \"#F0E442\", \"#0072B2\", \"#D55E00\", \"#CC79A7\")\nplot_df \u003c- as_tibble(combs) %\u003e%\n  gather(bcm:rue, key = \"Method\", value = \"Milliseconds (log scale)\")\n```\n\nFirst, we plot the results letting the `x` axis be the number of covariates `p`. For each facet, `n` is fixed. \n```{r}\nplot_df %\u003e%\n  ggplot(aes(x = p, y = `Milliseconds (log scale)`)) +\n  geom_line(aes(color = Method)) +\n  facet_grid(n~.,labeller = labeller(.rows = label_both, .cols = label_both)) +\n  scale_y_continuous(trans = \"log2\") +\n  scale_color_manual(values = cbPalette)\n```\n\nWhat the figure is showing is that:\n\n1. using the `rmvn_rue()` function, specifically tailored for this specific posterior distribution, is always better than computing `mu` and `Sigma` and then drawing from the posterior\n2. `rmvn_rue()` and `mgcv::rmvn()` are unaffected across facets, i.e. the sample size does not affect the computational time\n3. if `n` is small relative to `p`, the `rmvn_bcm()` function offers a substantial speed improvement\n\nIn the second figure, we instead fix `p` and let the `x` axis be the sample size `n`:\n```{r}\nplot_df %\u003e%\n  ggplot(aes(x = n, y = `Milliseconds (log scale)`)) +\n  geom_line(aes(color = Method)) +\n  facet_grid(p~.,labeller = labeller(.rows = label_both, .cols = label_both)) +\n  scale_y_continuous(trans = \"log2\") +\n  scale_color_manual(values = cbPalette)\n\n```\n\nThe second figure reiterates the point that you can always do better than naive sampling where `mu` and `Sigma` are computed explicitly. The take-away message is that notable speed improvements can be obtained by using one of the two sampling routines offered in the package when sampling from normal posterior distributions. If `p\u003en`, the `rmvn_bcm` function is the more faster alternative whereas `rmvn_rue` is faster otherwise.\n\n## Example 3: Bayesian linear regression\n\nTo see what the gains are in estimating a model, the package RcppDist provides an example of standard Bayesian linear regression. In standard Bayesian linear regression, the posterior mean and variance is constant and can be pre-computed outside of the MCMC loop. However, using scale mixtures and other hierarchical priors (e.g. normal-gamma, horseshoe, Dirichlet-Laplace, etc) the diagonal `D` matrix changes at every iteration, and as such the moments of the conditional posterior can no longer be pre-computed. To mimic this situation but without introducing unnecessary details, we will use the RcppDist linear regression example but fix the error variance to 1 and with the alteration that we compute `mu` and `Sigma` at every iteration (as you would need to do with a hierarchical prior). The RcppDist example with this modification is:\n\n```{Rcpp bayeslm, cache = TRUE}\n#include \u003cRcppArmadillo.h\u003e\n#include \u003cmvnorm.h\u003e\n// [[Rcpp::depends(RcppArmadillo, RcppDist)]]\n// [[Rcpp::export]]\narma::mat bayeslm(const arma::vec\u0026 y, const arma::mat\u0026 x,\n                   const int iters = 1000) {\n  int p = x.n_cols;\n  arma::vec d = arma::vec(p, arma::fill::ones); // prior variance is 1\n  arma::mat xtx, Sigma, mu;\n  \n  // Storage\n  arma::mat beta_draws(iters, p); // Object to store beta draws in\n  for ( int iter = 0; iter \u003c iters; ++iter ) {\n    xtx = x.t() * x; // X'X\n    xtx.diag() += arma::pow(d, -1.0); // add D^{-1}\n    \n    Sigma = xtx.i(); // the inverse is Sigma\n    mu = Sigma * x.t() * y; // compute mu\n    \n    beta_draws.row(iter) = rmvnorm(1, mu, Sigma);\n  }\n  return beta_draws;\n}\n```\nIn the code, the diagonal of `D` is taken to be 1, implying that we have a standard normal prior on all of the regression parameters.\n\nTo see what use of the `bayesnorm` means in terms of efficiency in this situation, we can create a similar function where we use `mvn_rue()` to sample from the posterior:\n```{Rcpp  bayeslm_rue, cache = TRUE}\n#include \u003cRcppArmadillo.h\u003e\n#include \u003cbayesnorm.h\u003e\n// [[Rcpp::depends(RcppArmadillo, bayesnorm)]]\n// [[Rcpp::export]]\narma::mat bayeslm_rue(const arma::vec\u0026 y, const arma::mat\u0026 x,\n                       const int iters = 1000) {\n  int p = x.n_cols;\n  arma::vec d = arma::vec(p, arma::fill::ones);\n  arma::mat beta_draws(p, iters);\n  for ( int iter = 0; iter \u003c iters; ++iter ) {\n    beta_draws.col(iter) = mvn_rue(x, d, y);\n  }\n  return beta_draws;\n}\n```\n\nWe will try a sample size of 500 and 100 covariates:\n```{r, cache = TRUE}\nn \u003c- 500\np \u003c- 100\n\nX \u003c- matrix(rnorm(n * p), n, p)\ny \u003c- matrix(rnorm(n), n, 1)\n\nmicrobenchmark::microbenchmark(bayeslm(y, X), bayeslm_rue(y, X), times = 10)\n```\n\nUsing the `mvn_rue()` function yields about a modest 10% speed improvement. \n\nIf we instead study the `p\u003en` case, improvements are more sizable. Create the same MCMC function but now using `mvn_bcm()` for sampling:\n```{Rcpp  bayeslm_bcm, cache = TRUE}\n#include \u003cRcppArmadillo.h\u003e\n#include \u003cbayesnorm.h\u003e\n// [[Rcpp::depends(RcppArmadillo, bayesnorm)]]\n// [[Rcpp::export]]\narma::mat bayeslm_bcm(const arma::vec\u0026 y, const arma::mat\u0026 x,\n                       const int iters = 1000) {\n  int p = x.n_cols;\n  arma::vec d = arma::vec(p, arma::fill::ones);\n  arma::mat beta_draws(p, iters);\n  for ( int iter = 0; iter \u003c iters; ++iter ) {\n    beta_draws.col(iter) = mvn_bcm(x, d, y);\n  }\n  return beta_draws;\n}\n```\n\nSetting the sample size to 50 and keeping 100 as the number of covariates shows a more impressive improvement in computational efficiency:\n```{r, cache = TRUE}\nn \u003c- 50\np \u003c- 100\n\nX \u003c- matrix(rnorm(n * p), n, p)\ny \u003c- matrix(rnorm(n), n, 1)\n\nmicrobenchmark::microbenchmark(bayeslm(y, X), bayeslm_bcm(y, X), times = 10)\n```\n\n## Incorporation into other packages\n\nTo use the sampling routines in other RcppArmadillo-based packages, all that is needed is to:\n\n- add `bayesnorm` in the `LinkingTo` field in the `DESCRIPTION`\n- add `#include \u003cbayesnorm.h\u003e` at the top of the file calling the functions (or in the package's header file)\n- the C++ functions are named `mvn_bcm` and `mvn_rue`\n\n\n### References\n\nBhattacharya, A., Chakraborty, A. and Mallick, B. (2016) Fast \nsampling with Gaussian scale mixture priors in high-dimensional regression,\n*Biometrika*, 103(4):985-991, [doi:10.1093/biomet/asw042](https://doi.org/10.1093/biomet/asw042)\n\nRue, H. (2001) Fast sampling of Gaussian Markov random fiels, *Journal of the Royal Statistical Society: Series B*, 63, 325-339, [doi:10.1111/1467-9868.00288](https://doi.org/10.1111/1467-9868.00288)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fankargren%2Fbayesnorm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fankargren%2Fbayesnorm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fankargren%2Fbayesnorm/lists"}