{"id":19120453,"url":"https://github.com/juliagaussianprocesses/bayesianlinearregressors.jl","last_synced_at":"2025-07-22T01:06:03.547Z","repository":{"id":43483526,"uuid":"176782111","full_name":"JuliaGaussianProcesses/BayesianLinearRegressors.jl","owner":"JuliaGaussianProcesses","description":"Bayesian Linear Regression in Julia","archived":false,"fork":false,"pushed_at":"2022-12-16T12:17:43.000Z","size":79,"stargazers_count":30,"open_issues_count":2,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-02-22T12:44:43.809Z","etag":null,"topics":["bayesian-inference","machine-learning","regression"],"latest_commit_sha":null,"homepage":"","language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JuliaGaussianProcesses.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-20T17:15:46.000Z","updated_at":"2024-08-27T12:23:07.000Z","dependencies_parsed_at":"2022-08-23T07:50:21.554Z","dependency_job_id":null,"html_url":"https://github.com/JuliaGaussianProcesses/BayesianLinearRegressors.jl","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/JuliaGaussianProcesses/BayesianLinearRegressors.jl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaGaussianProcesses%2FBayesianLinearRegressors.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaGaussianProcesses%2FBayesianLinearRegressors.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaGaussianProcesses%2FBayesianLinearRegressors.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaGaussianProcesses%2FBayesianLinearRegressors.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JuliaGaussianProcesses","download_url":"https://codeload.github.com/JuliaGaussianProcesses/BayesianLinearRegressors.jl/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaGaussianProcesses%2FBayesianLinearRegressors.jl/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266405405,"owners_count":23923536,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-21T11:47:31.412Z","response_time":64,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian-inference","machine-learning","regression"],"created_at":"2024-11-09T05:14:10.831Z","updated_at":"2025-07-22T01:06:03.524Z","avatar_url":"https://github.com/JuliaGaussianProcesses.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Bayesian Linear Regression in Julia\n\n[![Build Status](https://github.com/willtebbutt/BayesianLinearRegressors.jl/workflows/CI/badge.svg)](https://github.com/willtebbutt/BayesianLinearRegressors.jl/actions)\n[![Codecov](https://codecov.io/gh/willtebbutt/BayesianLinearRegressors.jl/branch/master/graph/badge.svg)](https://codecov.io/gh/JuliaGaussianProcesses/BayesianLinearRegressors.jl?branch=master)\n\nThis is a simple package that does one thing, Bayesian Linear Regression, in around 100 lines of code.\n\nIt _is_ actively maintained, but it might appear inactive as it's one of those packages which requires very little maintenance because it's very simple.\n\n## Intended Use and Functionality\n\nThe interface sits at roughly the same level as that of [Distributions.jl](https://github.com/JuliaStats/Distributions.jl/). This means that while you won't find a scikit-learn-style `fit` function, you will find all of the primitives that you need to construct such a function to suit your particular problem. In particular, one can:\n\n- Construct a `BayesianLinearRegressor` (BLR) object by providing a mean-vector and precision matrix for the weights of said regressor. This object represents a distribution over (linear) functions.\n- Use a `BayesianLinearRegressor` as an `AbstractGP`, as it implements the primary AbstractGP API.\n- Think of an instance of `BayesianLinearRegressor` as a very restricted GP, where the time complexity of inference scales linearly in the number of observations `N`.\n- Draw function samples from a `BayesianLinearRegressor` using `rand`.\n- Construct a `BasisFunctionRegressor` object which is a thin wrapper around a `BayesianLinearRegressor` to allow a non-linear feature mapping `ϕ` to act on the input.\n\n## Conventions\n\n`BayesianLinearRegressors` is consistent with `AbstractGPs`.\nConsequently, a `BayesianLinearRegressor` in `D` dimensions can work with the following input types:\n1. `ColVecs` -- a wrapper around an `D x N` matrix of `Real`s saying that each column should be interpreted as an input.\n2. `RowVecs`s -- a wrapper around an `N x D` matrix of `Real`s, saying that each row should be interpreted as an input.\n3. `Matrix{\u003c:Real}` -- must be `D x N`. Prefer using `ColVecs` or `RowVecs` for the sake of being explicit.\n\nConsult the `Design` section of the [KernelFunctions.jl](https://juliagaussianprocesses.github.io/KernelFunctions.jl/dev/design/) docs for more info on these conventions.\n\nOutputs for a BayesianLinearRegressor should be an `AbstractVector{\u003c:Real}` of length `N`.\n\n## Example Usage\n\n\n```julia\n# Install the packages if you don't already have them installed\n] add AbstractGPs BayesianLinearRegressors LinearAlgebra Random Plots Zygote\nusing AbstractGPs, BayesianLinearRegressors, LinearAlgebra, Random, Plots, Zygote\n\n# Fix seed for re-producibility.\nrng = MersenneTwister(123456)\n\n# Construct a BayesianLinearRegressor prior over linear functions of `X`.\nmw, Λw = zeros(2), Diagonal(ones(2))\nf = BayesianLinearRegressor(mw, Λw)\n\n# Index into the regressor and assume heteroscedastic observation noise `Σ_noise`.\nN = 10\nX = ColVecs(hcat(range(-5.0, 5.0, length=N), ones(N))')\nΣ_noise = Diagonal(exp.(randn(N)))\nfX = f(X, Σ_noise)\n\n# Generate some toy data by sampling from the prior.\ny = rand(rng, fX)\n\n# Compute the adjoint of `rand` w.r.t. everything given random sensitivities of y′.\n_, back_rand = Zygote.pullback(\n    (X, Σ_noise, mw, Λw)-\u003erand(rng, BayesianLinearRegressor(mw, Λw)(X, Σ_noise), 5),\n    X, Σ_noise, mw, Λw,\n)\nback_rand(randn(N, 5))\n\n# Compute the `logpdf`. Read as `the log probability of observing `y` at `X` under `f`, and\n# Gaussian observation noise with zero-mean and covariance `Σ_noise`.\nlogpdf(fX, y)\n\n# Compute the gradient of the `logpdf` w.r.t. everything.\nZygote.gradient(\n    (X, Σ_noise, y, mw, Λw)-\u003elogpdf(BayesianLinearRegressor(mw, Λw)(X, Σ_noise), y),\n    X, Σ_noise, y, mw, Λw,\n)\n\n# Perform posterior inference. Note that `f′` has the same type as `f`.\nf′ = posterior(fX, y)\n\n# Compute `logpdf` of the observations under the posterior predictive.\nlogpdf(f′(X, Σ_noise), y)\n\n# Sample from the posterior predictive distribution.\nN_plt = 1000\nX_plt = ColVecs(hcat(range(-6.0, 6.0, length=N_plt), ones(N_plt))')\n\n# Compute some posterior marginal statisics.\nnormals = marginals(f′(X_plt, eps()))\nm′X_plt = mean.(normals)\nσ′X_plt = std.(normals)\n\n# Plot the posterior. This uses the default AbstractGPs plotting recipes.\nposterior_plot = plot();\nplot!(posterior_plot, X_plt.X[1, :], f′(X_plt, eps()); color=:blue, ribbon_scale=3);\nsampleplot!(posterior_plot, X_plt.X[1, :], f′(X_plt, eps()); color=:blue, samples=10);\nscatter!(posterior_plot, X.X[1, :], y; # Observations.\n    markercolor=:red,\n    markershape=:circle,\n    markerstrokewidth=0.0,\n    markersize=4,\n    markeralpha=0.7,\n    label=\"\",\n);\ndisplay(posterior_plot);\n```\n\n## Basis Function Regression\n\nAny instance of a `BayesianLinearRegressor` can be replaced by a `BasisFunctionRegressor` (BFR). A `BasisFunctionRegressor` is a thin wrapper around a `BayesianLinearRegressor`, but includes a potentially non-linear feature mapping `ϕ` which is applied to the input before it is passed to the underlying BLR. It is essentially defined as `bfr(X) = blr(ϕ(X))`.\n\n``` julia\nusing AbstractGPs, BayesianLinearRegressors, LinearAlgebra\n\nX = RowVecs(hcat(range(-1.0, 1.0, length=5)))\nblr = BayesianLinearRegressor(zeros(2), Diagonal(ones(2)))\n\n# N.B. ϕ must accept one of the allowed input types and\n# must return the same type (in this case RowVecs)\nϕ(x::RowVecs) = RowVecs(hcat(ones(length(x)), prod.(x)))\n\nbfr = BasisFunctionRegressor(blr, ϕ)\n\n# These are equivalent\nvar(bfr(X)) == var(blr(ϕ(X)))\n```\n\n## Drawing Function Samples\n\nThere are two ways of drawing samples from `f::Union{BayesianLinearRegressor,BasisFunctionRegressor}`. The first is by using the `AbstractGPs` API (as in the above examples) where a `FiniteBLR` projection is first produced at a fixed set of input locations `X` with some specified observation noise `Σy`: `fx = f(X, Σy)::FiniteBLR`. Then, observations (including noise) at `X` can be sampled directly with `rand(rng, fx)`.\nThe other way is to draw a sample of an entire function from `f` using `g = rand(rng, f)`. This samples a value for the weights `w ~ N(mw, Λw)` and produces a function `g(X) = w'X` which can be evaluated at any input locations. Note that this method corresponds to drawing samples of noiseless observations.\n\n``` julia\nusing AbstractGPs, BayesianLinearRegressors, LinearAlgebra, Random, Plots\n\n# Fix seed for re-producibility.\nrng = MersenneTwister(123456)\n\nX = RowVecs(hcat(range(-1.0, 1.0, length=5)))\nf = BayesianLinearRegressor(zeros(2), Diagonal(ones(2)))\n\n## The first method of drawing samples - using the AbstractGPs API:\n# Index into the regressor at fixed inputs X and assume homoscedastic observation noise `Σ_noise`.\nN = 10\nX = ColVecs(hcat(range(-5.0, 5.0, length=N), ones(N))')\nΣ_noise = 0.1\nfX = f(X, Σ_noise)\n\nrand(rng, fX)\n\n## The second method - sampling an entire function:\ng = rand(rng, f)\n\n# This sample can now be evaulated at any input locations\ng(X)\n\nX′ = ColVecs(hcat(range(10.0, 15.0, length=N), ones(N))')\ng(X′)\n\n# Sample multiple functions for plotting\ngs = rand(rng, f, 100)\n\nN_plt = 1000\nX_plt = ColVecs(hcat(range(-6.0, 6.0, length=N_plt), ones(N_plt))')\ny_plt = reduce(hcat, [g(X_plt) for g in gs])\n\nsample_plot = plot();\nplot!(sample_plot, X_plt.X[1,:], y_plt;\n    label=\"\",\n    color=:black,\n    linealpha=0.4,\n);\ndisplay(sample_plot)\n```\n\n## Up For Grabs\n\n- Scikit-learn style interface: it wouldn't be too hard to implement a scikit-learn - style interface to handle basic regression tasks, so please feel free to make a PR that implements this.\n- Monte Carlo VI (MCVI): i.e. variational inference using the reparametrisation trick. This could be very useful when working with large data sets and applying big non-linear transformations, such as neural networks, to the inputs as it would enable mini-batching. I would envisage at least supporting both a dense approximate posterior covariance and diagonal (i.e. mean-field), where the former is for small-moderate dimensionalities and the latter for very high-dimensional problems.\n\n## Bugs, Issues, and PRs\n\nPlease do report any bugs you find by raising an issue. Please also feel free to raise PRs, especially if for one of the above `Up For Grabs` items. Raise an issue to discuss the extension in detail before opening a PR if you prefer, though.\n\n\n## Related Work\n\n[BayesianLinearRegression.jl](https://github.com/cscherrer/BayesianLinearRegression.jl) is closely related, but appears to be a WIP and hasn't been touched in around a year or so (as of 27-03-2019).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuliagaussianprocesses%2Fbayesianlinearregressors.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjuliagaussianprocesses%2Fbayesianlinearregressors.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuliagaussianprocesses%2Fbayesianlinearregressors.jl/lists"}