https://github.com/locuslab/orthogonal-convolutions

Implementations of orthogonal and semi-orthogonal convolutions in the Fourier domain with applications to adversarial robustness
https://github.com/locuslab/orthogonal-convolutions

Last synced: 3 months ago
JSON representation

Implementations of orthogonal and semi-orthogonal convolutions in the Fourier domain with applications to adversarial robustness

• Host: GitHub
• URL: https://github.com/locuslab/orthogonal-convolutions
• Owner: locuslab
• License: mit
• Created: 2021-03-13T23:06:11.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2021-04-09T17:22:09.000Z (over 3 years ago)
• Last Synced: 2024-07-12T12:41:05.348Z (4 months ago)
• Language: Jupyter Notebook
• Homepage:
• Size: 307 KB
• Stars: 40
• Watchers: 4
• Forks: 7
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-normalizing-flows - [Code

README

        
# Orthogonalizing Convolutional Layers with the Cayley Transform



This repository contains implementations and source code to reproduce experiments

for the ICLR 2021 spotlight paper Orthogonalizing Convolutional Layers with the Cayley Transform

by Asher Trockman and Zico Kolter.




Check out our tutorial on

FFT-based convolutions and how to orthogonalize them

in this Jupyter notebook.



*(more information and code coming soon)*

### Getting Started

You can clone this repo using:

```

git clone https://github.com/locuslab/orthogonal-convolutions --recursive

```

where the `--recursive` is necessary for the submodules.

The most important dependency is **PyTorch >= 1.8**. If you like, you can set up a new conda environment:

```

conda create --name orthoconv python=3.6

conda activate orthoconv

conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge

conda install --file requirements.txt

```

### The Orthogonal Convolutional Layer

Our orthogonal convolutional layer can be found in `layers.py`. The actual layer is the module `CayleyConv`. It depends on the function `cayley`, our implementation of the Cayley transform for (semi-)orthogonalization. Additionally, `CayleyConv` is a subclass of `StridedConv`, which emulates striding functionality by reshaping the input tensor.

### Running Experiments

The script `train.py` can be used to run most of the experiments from our paper. To try the "flagship" experiment demonstrating better clean accuracy and -norm-bounded deterministic certifiable robustness, run:

```

python train.py --epochs=200 --conv=CayleyConv --linear=CayleyLinear

```

To compare with BCOP as in our paper, run:

```

python train.py --epochs=200 --conv=BCOP --linear=BjorckLinear

```