https://github.com/sophiaas/spectral-universality

Official PyTorch and JAX Implementation of "Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networks"
https://github.com/sophiaas/spectral-universality

fourier geometric-deep-learning group-theory invariance

Last synced: about 12 hours ago
JSON representation

Official PyTorch and JAX Implementation of "Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networks"

• Host: GitHub
• URL: https://github.com/sophiaas/spectral-universality
• Owner: sophiaas
• Created: 2023-04-25T17:46:46.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-02-01T18:08:06.000Z (12 months ago)
• Last Synced: 2024-07-03T09:13:57.455Z (7 months ago)
• Topics: fourier, geometric-deep-learning, group-theory, invariance
• Language: Python
• Homepage: https://arxiv.org/abs/2312.08550
• Size: 445 KB
• Stars: 24
• Watchers: 2
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Harmonics of Learning







## Description

This repository accompanies the paper:

Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networks (2024)

Here, we provide an implementation of a complex-valued Power-Spectral Network trained via contrastive learning on an invariance objective for a finite group. As shown in the companion paper, at convergence the network learns all the irreducible unitary representations of the group. In particular, the multiplication table can be extracted from its weights.

We provide implementations of the model and its training in both `PyTorch` and `JAX`.

## Setup

```

python 3.8+

pip install -r requirements.txt

```

## Groups

The file `groups.py` provides implementations of various finite groups, including cyclic, dihedral and symmetric.

## Training

In order to train the models in `PyTorch` and in `JAX`, run the files `train_torch.py` and `train_JAX.py` respectively. The training parameters are set at the beginning of these files.