https://github.com/YihongDong/FAN

https://github.com/YihongDong/FAN

Last synced: about 2 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/YihongDong/FAN
• Owner: YihongDong
• License: mit
• Created: 2024-10-03T14:31:12.000Z (3 months ago)
• Default Branch: main
• Last Pushed: 2024-11-19T17:08:33.000Z (about 2 months ago)
• Last Synced: 2024-11-19T17:28:46.338Z (about 2 months ago)
• Language: Python
• Size: 12.8 MB
• Stars: 96
• Watchers: 4
• Forks: 8
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• StarryDivineSky - YihongDong/FAN

README

        
# FAN: Fourier Analysis Networks

[**Paper**](https://arxiv.org/abs/2410.02675)



|                           | MLP Layer                                                |  FAN layer                                                                 |

|---------------------------|:----------------------------------------------------------:|:------------------------------------------------------------------------:|

| **Formula**                | $\Phi(x) = \sigma(B_{m} + W_{m}x)$                       | $\phi(x) = [\cos(W_px)\|\| \sin(W_px)\|\| \sigma(B_{\bar{p}} + W_{\bar{p}}x)]$       |

| **Num of Params**          | $(d_\text{input} \times d_\text{output}) + d_\text{output}$ | $(1-\frac{d_p}{d_\text{output}})\times((d_\text{input} \times d_\text{output}) + d_\text{output})$ |

| **FLOPs**                  | $2\times(d_\text{input} \times d_\text{output})$
 $+ d_\text{output} \times \text{FLOPs}_\text{non-linear}$ | $(1-\frac{d_p}{d_\text{output}})\times(2\times(d_\text{input} \times d_\text{output}))$ 
 $+ d_\text{output} \times \text{FLOPs}_\text{non-linear} $ |

## Periodicity Modeling

```shell

cd Periodicity_Modeling

bash ./run.sh

```

![sin](./img/sin.jpg)

![mod](./img/mod.jpg)

## Sentiment Analysis

The data can be automatically downloaded using the Huggingface Datasets `load_dataset` function in the `./Sentiment_Analysis/get_dataloader.py`. 

```shell

cd Sentiment_Analysis

bash scripts/Trans_with_FAN/train_ours.sh

bash scripts/Trans_with_FAN/test_ours.sh

```

## Timeseries Forecasting

You can obtain data from [Google Drive](https://drive.google.com/drive/folders/1v1uLx5zhGaNAOTIqHLHYMXtA-XFrKTxS?usp=sharing). All the datasets are well pre-processed and can be used easily.

```shell

cd Timeseries_Forecasting

bash scripts/Weather_script/Modified_Transformer.sh 

```

## Symbolic Formula Representation

```shell

cd Symbolic_Formula_Representation

python gen_dataset.py

bash run_train_fan.sh

```

## Image Recognition

```shell

cd Image_Recognition

bash run_image_recognition.sh

```

We update the results on image recognition tasks and the detailed setup can be found in the Appendix of our latest-version paper.

![Image recognition tasks](./img/IR.jpg)

## Citation

```

@article{dong2024fan,

  title={FAN: Fourier Analysis Networks},

  author={Yihong Dong and Ge Li and Yongding Tao and Xue Jiang and Kechi Zhang and Jia Li and Jing Su and Jun Zhang and Jingjing Xu},

  journal={arXiv preprint arXiv:2410.02675},

  year={2024}

}

```