{"id":37671158,"url":"https://github.com/aaltoml/sfr-experiments","last_synced_at":"2026-01-16T12:04:33.709Z","repository":{"id":227686743,"uuid":"760407387","full_name":"AaltoML/sfr-experiments","owner":"AaltoML","description":"Code accompanying ICLR 2024 paper \"Function-space Parameterization of Neural Networks for Sequential Learning\"","archived":false,"fork":false,"pushed_at":"2024-05-15T13:41:21.000Z","size":2244,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-09-08T22:18:38.160Z","etag":null,"topics":["bayesian-deep-learning","bayesian-inference","bayesian-neural-networks","deep-learning","gaussian-processes","laplace-approximation","pytorch"],"latest_commit_sha":null,"homepage":"https://aaltoml.github.io/sfr/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AaltoML.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-20T11:12:06.000Z","updated_at":"2024-05-17T09:41:01.000Z","dependencies_parsed_at":"2024-03-15T20:44:44.170Z","dependency_job_id":"674133f1-a5c5-42e7-89fb-f70e1d787000","html_url":"https://github.com/AaltoML/sfr-experiments","commit_stats":null,"previous_names":["aaltoml/sfr-experiments"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/AaltoML/sfr-experiments","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AaltoML%2Fsfr-experiments","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AaltoML%2Fsfr-experiments/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AaltoML%2Fsfr-experiments/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AaltoML%2Fsfr-experiments/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AaltoML","download_url":"https://codeload.github.com/AaltoML/sfr-experiments/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AaltoML%2Fsfr-experiments/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28478427,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T11:59:17.896Z","status":"ssl_error","status_checked_at":"2026-01-16T11:55:55.838Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian-deep-learning","bayesian-inference","bayesian-neural-networks","deep-learning","gaussian-processes","laplace-approximation","pytorch"],"created_at":"2026-01-16T12:04:33.095Z","updated_at":"2026-01-16T12:04:33.699Z","avatar_url":"https://github.com/AaltoML.png","language":"Jupyter Notebook","readme":"# Function-space Parameterization of Neural Networks for Sequential Learning\nCode accompanying ICLR 2024 submission *Function-space Parameterization of Neural Networks for Sequential Learning*.\nThis repository contains code for reproducing the experiments in the ICLR 2024 paper.\nPlease see [this repo](https://github.com/AaltoML/sfr/tree/main) for a clean and minimal implementation of Sparse Function-space Representation of Neural Networks (SFR).\nWe recommend using the [clean and minimal repo](https://github.com/AaltoML/sfr/tree/main).\n\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\n            \u003ca href=\"https://openreview.net/forum?id=2dhxxIKhqz\u0026referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DICLR.cc%2F2024%2FConference%2FAuthors%23your-submissions)\"\u003e\n              \u003cstrong \u003eFunction-space Parameterization of Neural Networks for Sequential Learning\u003c/strong\u003e\u003cbr\u003e\n            \u003c/a\u003e\n            Aidan Scannell*, Riccardo Mereu*, Paul Chang, Ella Tamir, Joni Pajarinen, Arno Solin\u003cbr\u003e\n            \u003cstrong\u003eInternational Conference on Learning Representations (ICLR 2024)\u003c/strong\u003e\u003cbr\u003e\n            \u003c!-- \u003ca href=\"https://arxiv.org/abs/2309.02195\"\u003e\u003cimg alt=\"Paper\" src=\"https://img.shields.io/badge/-Paper-gray\"\u003e\u003c/a\u003e --\u003e\n            \u003c!-- \u003ca href=\"https://github.com/aidanscannell/sfr\"\u003e\u003cimg alt=\"Code\" src=\"https://img.shields.io/badge/-Code-gray\" \u003e\u003c/a\u003e --\u003e\n        \u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\n            \u003ca href=\"https://arxiv.org/abs/2309.02195\"\u003e\n              \u003cstrong\u003eSparse Function-space Representation of Neural Networks\u003c/strong\u003e\u003cbr\u003e\n            \u003c/a\u003e\n            Aidan Scannell*, Riccardo Mereu*, Paul Chang, Ella Tamir, Joni Pajarinen, Arno Solin\u003cbr\u003e\n            \u003cstrong\u003eICML 2023 Workshop on Duality Principles for Modern Machine Learning\u003c/strong\u003e\u003cbr\u003e\n            \u003c!-- \u003ca href=\"https://arxiv.org/abs/2309.02195\"\u003e\u003cimg alt=\"Paper\" src=\"https://img.shields.io/badge/-Paper-gray\"\u003e\u003c/a\u003e --\u003e\n            \u003c!-- \u003ca href=\"https://github.com/aidanscannell/sfr\"\u003e\u003cimg alt=\"Code\" src=\"https://img.shields.io/badge/-Code-gray\" \u003e\u003c/a\u003e --\u003e\n        \u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\n\n## Install\n\n### Install using virtual environment\nMake a virtual environment:\n``` sh\npython -m venv .venv\n```\nActivate it with:\n``` sh\nsource .venv/bin/activate\n```\nInstall the dependencies with:\n``` sh\npython -m pip install --upgrade pip\npip install laplace-torch==0.1a2\npip install -e \".[experiments]\"\n```\nWe install `laplace-torch` separately due to version conflicts with `backpacpk-for-pytorch`.\nNote that `laplace-torch` is only used for running the baselines.\n\n### Install using pip\nAlternatively, manually install the dependencies with:\n``` sh\npip install laplace-torch==0.1a2\npip install -r requirements.txt\n```\n\n## Reproducing experiments\nSee [experiments](./experiments/) for details on how to reproduce the results in the paper.\nThis includes code for generating the tables and figures.\n\n## Useage\nSee the [notebooks/README.md](./notebooks) for how to use our code for both regression and classification.\n\n### Example\nHere's a short example:\n```python\nimport src\nimport torch\n\ntorch.set_default_dtype(torch.float64)\n\ndef func(x, noise=True):\n    return torch.sin(x * 5) / x + torch.cos(x * 10)\n\n# Toy data set\nX_train = torch.rand((100, 1)) * 2\nY_train = func(X_train, noise=True)\ndata = (X_train, Y_train)\n\n# Training config\nwidth = 64\nnum_epochs = 1000\nbatch_size = 16\nlearning_rate = 1e-3\ndelta = 0.00005  # prior precision\ndata_loader = torch.utils.data.DataLoader(\n    torch.utils.data.TensorDataset(*data), batch_size=batch_size\n)\n\n# Create a neural network\nnetwork = torch.nn.Sequential(\n    torch.nn.Linear(1, width),\n    torch.nn.Tanh(),\n    torch.nn.Linear(width, width),\n    torch.nn.Tanh(),\n    torch.nn.Linear(width, 1),\n)\n\n# Instantiate SFR (handles NN training/prediction as they're coupled via the prior/likelihood)\nsfr = src.sfr.SFR(\n    network=network,\n    prior=src.priors.Gaussian(params=network.parameters, delta=delta),\n    likelihood=src.likelihoods.Gaussian(sigma_noise=2),\n    output_dim=1,\n    num_inducing=32,\n    dual_batch_size=None, # this reduces the memory required for computing dual parameters\n    jitter=1e-4,\n)\n\nsfr.train()\noptimizer = torch.optim.Adam([{\"params\": sfr.parameters()}], lr=learning_rate)\nfor epoch_idx in range(num_epochs):\n    for batch_idx, batch in enumerate(data_loader):\n        x, y = batch\n        loss = sfr.loss(x, y)\n        optimizer.zero_grad()\n        loss.backward()\n        optimizer.step()\n\nsfr.set_data(data) # This builds the dual parameters\n\n# Make predictions in function space\nX_test = torch.linspace(-0.7, 3.5, 300, dtype=torch.float64).reshape(-1, 1)\nf_mean, f_var = sfr.predict_f(X_test)\n\n# Make predictions in output space\ny_mean, y_var = sfr.predict(X_test)\n```\n\n\n## Citation\nPlease consider citing our ICLR 2024 paper.\n```bibtex\n@inproceedings{scannellFunction2024,\n  title           = {Function-space Prameterization of Neural Networks for Sequential Learning},\n  booktitle       = {Proceedings of The Twelth International Conference on Learning Representations (ICLR 2024)},\n  author          = {Aidan Scannell and Riccardo Mereu and Paul Chang and Ella Tami and Joni Pajarinen and Arno Solin},\n  year            = {2024},\n  month           = {5},\n}\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaltoml%2Fsfr-experiments","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faaltoml%2Fsfr-experiments","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaltoml%2Fsfr-experiments/lists"}