https://github.com/AvivNavon/DWSNets

Official implementation for Equivariant Architectures for Learning in Deep Weight Spaces [ICML 2023]
https://github.com/AvivNavon/DWSNets

Last synced: 9 days ago
JSON representation

Official implementation for Equivariant Architectures for Learning in Deep Weight Spaces [ICML 2023]

• Host: GitHub
• URL: https://github.com/AvivNavon/DWSNets
• Owner: AvivNavon
• License: mit
• Created: 2023-02-12T11:12:45.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2023-08-21T07:00:37.000Z (about 1 year ago)
• Last Synced: 2024-08-01T13:22:31.137Z (3 months ago)
• Language: Jupyter Notebook
• Homepage: https://avivnavon.github.io/DWSNets/
• Size: 1.45 MB
• Stars: 78
• Watchers: 5
• Forks: 10
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# DWSNets

Official implementation for [_Equivariant Architectures for Learning in Deep Weight Spaces_](https://arxiv.org/abs/2301.12780) by Aviv Navon, Aviv Shamsian, Idan Achituve, Ethan Fetaya, Gal Chechik, Haggai Maron.

 

    



Our implementation follows the block structure as described in the paper.

![](misc/blocks.png)

## Setup environment

To run the experiments, first create clean virtual environment and install the requirements.

```bash

conda create -n dwsnets python=3.9

conda activate dwsnets

conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch

```

Install the repo:

```bash

git clone https://github.com/AvivNavon/DWSNets.git

cd DWSNets

pip install -e .

```

## Introduction Notebook

An introduction notebook for INR classification with DWSNets: 

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/AvivNavon/DWSNets/blob/main/notebooks/mnist-inr-classification.ipynb)

## Run experiment

To run specific experiment, please follow the instructions in the README file within each experiment folder. 

It provides full instructions and details for downloading the data and reproducing the results reported in the paper.

### Dataset 

**The datasets are available [here](https://www.dropbox.com/sh/56pakaxe58z29mq/AABtWNkRYroLYe_cE3c90DXVa?dl=0).**

- [Sine regression INRs](https://www.dropbox.com/sh/56pakaxe58z29mq/AABtWNkRYroLYe_cE3c90DXVa?dl=0&preview=sine-regression-inrs.zip)

- [MNIST INRs](https://www.dropbox.com/sh/56pakaxe58z29mq/AABtWNkRYroLYe_cE3c90DXVa?dl=0&preview=mnist-inrs.zip)

- [SSL INRs](https://www.dropbox.com/sh/56pakaxe58z29mq/AABtWNkRYroLYe_cE3c90DXVa?dl=0&preview=ssl-inrs.zip)

- [CIFAR10 classifiers](https://www.dropbox.com/sh/56pakaxe58z29mq/AABtWNkRYroLYe_cE3c90DXVa?dl=0&preview=cifar_clf_models.zip)

## Citation

If you find our work or this code to be useful in your own research, please consider citing the following paper:

```bib

@InProceedings{pmlr-v202-navon23a,

  title = 	 {Equivariant Architectures for Learning in Deep Weight Spaces},

  author =       {Navon, Aviv and Shamsian, Aviv and Achituve, Idan and Fetaya, Ethan and Chechik, Gal and Maron, Haggai},

  booktitle = 	 {Proceedings of the 40th International Conference on Machine Learning},

  pages = 	 {25790--25816},

  year = 	 {2023},

  editor = 	 {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan},

  volume = 	 {202},

  series = 	 {Proceedings of Machine Learning Research},

  month = 	 {23--29 Jul},

  publisher =    {PMLR},

  pdf = 	 {https://proceedings.mlr.press/v202/navon23a/navon23a.pdf},

  url = 	 {https://proceedings.mlr.press/v202/navon23a.html},

}

```