{"id":13738256,"url":"https://github.com/gatheluck/FourierHeatmap","last_synced_at":"2025-05-08T16:33:00.857Z","repository":{"id":40967507,"uuid":"248454129","full_name":"gatheluck/FourierHeatmap","owner":"gatheluck","description":"Unofficial pytorch implementation of Fourier Heat Map proposed in 'A Fourier Perspective on Model Robustness in Computer Vision' [Yin+, NeurIPS2019]","archived":false,"fork":false,"pushed_at":"2024-05-03T19:54:52.000Z","size":596,"stargazers_count":69,"open_issues_count":7,"forks_count":12,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-11-15T07:34:03.100Z","etag":null,"topics":["fourier-analysis","heatmap","pytorch","robustness"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gatheluck.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-19T08:54:02.000Z","updated_at":"2024-10-14T02:30:32.000Z","dependencies_parsed_at":"2024-05-03T21:04:33.439Z","dependency_job_id":null,"html_url":"https://github.com/gatheluck/FourierHeatmap","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gatheluck%2FFourierHeatmap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gatheluck%2FFourierHeatmap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gatheluck%2FFourierHeatmap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gatheluck%2FFourierHeatmap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gatheluck","download_url":"https://codeload.github.com/gatheluck/FourierHeatmap/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253105413,"owners_count":21855019,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fourier-analysis","heatmap","pytorch","robustness"],"created_at":"2024-08-03T03:02:16.119Z","updated_at":"2025-05-08T16:33:00.541Z","avatar_url":"https://github.com/gatheluck.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# FourierHeatmap (latest release: v0.2.0)\n\n[![CI](https://github.com/gatheluck/FourierHeatmap/workflows/CI/badge.svg)](https://github.com/gatheluck/FourierHeatmap/actions?query=workflow%3ACI)\n[![codecov](https://codecov.io/gh/gatheluck/FourierHeatmap/branch/master/graph/badge.svg?token=17KIZNS046)](https://codecov.io/gh/gatheluck/FourierHeatmap)\n[![MIT License](https://img.shields.io/github/license/gatheluck/FourierHeatmap)](LICENSE)\n\nThis is an unofficial pytorch implementation of Fourier Heat Map which is proposed in the paper, [A Fourier Perspective on Model Robustness in Computer Vision [Yin+, NeurIPS2019]](https://arxiv.org/abs/1906.08988). \n\nFourier Heat Map allows to investigate the sensitivity of CNNs to high and low frequency corruptions via a perturbation analysis in the Fourier domain.\n\n\u003cimg src=\"samples/FourierHeatmap-Teaser.png\" height=\"300px\"\u003e\n\n## News\n- We release v0.2.0. API is renewed and some useful libraries (e.g. [hydra](https://hydra.cc/docs/intro/)) are added.\n\n- Previous version is still available as [v0.1.0](https://github.com/gatheluck/FourierHeatmap/tree/v0.1.0).\n\n- [Docker is supported](#Evaluating_Fourier_Heat_Map_through_Docker). Now, you can evaluate Fourier Heat Map on the Docker container.\n\n## Requirements\nThis library requires following as a pre-requisite.\n- python 3.9+\n- poetry\n\nNote that I run the code with Ubuntu 20, Pytorch 1.8.1, CUDA 11.0.\n\n## Installation\nThis repo uses [poetry](https://python-poetry.org/) as a package manager. \nThe following code will install all necessary libraries under `.venv/`.\n\n```\n$ git clone git@github.com:gatheluck/FourierHeatmap.git\n$ cd FourierHeatmap\n$ pip install poetry  # If you haven't installed poetry yet.\n$ poetry install\n```\n\n## Setup\n\n### Dataset\n\nThis codes expect datasets exist under `data/`. For example, if you want to evaluate Fourier Heat Map for ImageNet, please set up like follows:\n\n```\nFourierHeatmap\n├── data\n│\t└── imagenet\n│\t\t├── train/\n│\t\t└── val/\n```\n\n## Usage\n\n### Visualizing Fourier basis\n\nThe script `fhmap/fourier/basis.py` generates Fourier base functions. For example:\n\n```\n$ poetry run python fhmap/fourier/basis.py\n```\n\nwill generate 31x31 2D Fourier basis and save as an image under `outputs/basis.png`. The generated image should be like follows. \n\n\u003cimg src=\"samples/basis_31x31.png\" height=\"300px\"\u003e\n\n### Evaluating Fourier Heat Map\n\nThe script `fhmap/apps/eval_fhmap.py`\neveluate Fourier Heat Map for a model. For example:\n\n```\n$ poetry run python fhmap/apps/eval_fhmap.py dataset=cifar10 arch=resnet56 weightpath=[PYTORCH_MODEL_WEIGHT_PATH] eps=4.0\n```\n\nwill generate 31x31 Fourier Heat Map for ResNet56 on CIFAR-10 dataset and save as an image under `outputs/eval_fhmap/`. The generated image should be like follows. \n\n\u003cimg src=\"samples/cifar10_resnet56_natural.png\" height=\"300px\"\u003e\n\n\nNote that the L2 norm size (=eps) of Fourier basis use in original paper is following:\n| dataset | eps\n---- | ----\n| CIFAR-10 | 4.0\n| ImageNet | 15.7\n\n## Evaluating custom dataset and model\n\n\n\n### Evaluating your custom dataset \n\nIf you want to evaluate Fourier Heat Map on your custom dataset, please refer follwing instraction.\n\n- Implement `YourCustomDatasetStats` class: \n\t- This class holds basic dataset information.\n\t- `YourCustomDatasetStats` class should inherit from original `DatasetStats` class in `factory/dataset` module and also shoud be placed in `factory/dataset` module. \n\t- For details, please refer to the `Cifar10Stats` class in `factory/dataset` module.\n\n- Implement `YourCustomDataModule` class:\n\t- This class is responsible for preprocess, transform (includes adding Fourier Noise to image) and create test dataset.\n\t- `YourCustomDataModule` class should inherit from `BaseDataModule` class in `factory/dataset` module and also shoud be placed in `factory/dataset` module. \n\t- For details, please refer to the `Cifar10DataModule` class in `factory/dataset` module.\n\n- Implement `YourCustomDatasetConfig` class:\n\t- This class is needed for applying [hydra](https://hydra.cc/)'s [dynamic object instantiation](https://hydra.cc/docs/patterns/instantiate_objects/overview) to dataset class.\n\t- `YourCustomDatasetConfig` class should inherit from `DatasetConfig` class in `schema/dataset` module and also shoud be placed in `schema/dataset` module. Please add `YourCustomDatasetConfig` to `schema/__init__`.\n\t- For details, please refer to the `Cifar10Config` class in `schema/dataset` module.\n\n- Add option for your custom dataset:\n\t- Lastly, please add the config of your custom dataset to `ConfigStore` class by adding a follwing line to `apps/eval_fhmap`.\n\n\t```\n\tcs.store(group=\"dataset\", name=\"yourcustomdataset\", node=schema.YourCustomDatasetConfig)\n\t```\n\nNow, you will be able to call your custom dataset like following.\n\n```\n$ poetry run python fhmap/apps/eval_fhmap.py dataset=yourcustomdataset arch=resnet50 weightpath=[PYTORCH_MODEL_WEIGHT_PATH] eps=4.0\n```\n\n### Evaluating your custom architecture (model)\n\nIf you want to evaluate Fourier Heat Map on your custom architecture (model), please refer follwing instraction.\n\n- Implement `YourCustomArch` class:\n\t- Please implement class or function which return your custom architecture. The custom architecture have to subclass of `torch.nn.module`.\n\t- For details, please refer to the `factory/archs/resnet` module.\n\n- Implement `YourCustomArchConfig` class:\n\t- This class is needed for applying [hydra](https://hydra.cc/)'s [dynamic object instantiation](https://hydra.cc/docs/patterns/instantiate_objects/overview) to architecture class.\n\t- `YourCustomArchConfig` class should inherit from `ArchConfig` class in `schema/arch` module and also shoud be placed in `schema/arch` module. Please add `YourCustomArchConfig` to `schema/__init__`.\n\t- For details, please refer to the `Resnet56Config` class in `schema/arch` module.\n\t- If you want to use architectures which is provided by other libs like [pytorch](https://github.com/pytorch/pytorch) or [timm](https://github.com/rwightman/pytorch-image-models), please refere to the `Resnet50Config` class in `schema/arch` module.\n\n- Add option for your custom architecture:\n\t- Lastly, please add the config of your custom architecture to `ConfigStore` class by adding a follwing line to `apps/eval_fhmap`.\n\n\t```\n\tcs.store(group=\"arch\", name=\"yourcustomarch\", node=schema.YourCustomArchConfig)\n\t```\n\nNow, you will be able to call your custom arch like following.\n\n```\n$ poetry run python fhmap/apps/eval_fhmap.py dataset=cifar10 arch=yourcustomarch weightpath=[PYTORCH_MODEL_WEIGHT_PATH] eps=4.0\n```\n\n## Evaluating Fourier Heat Map through Docker\nIn order to use FourierHeatmap throgh docker, please install Docker with [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/overview.html) beforehand. For detail, please refere [official installation guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#installation-guide).\n\nIf `nvidia-smi` is able to run through docker like following, it is successfully installed.\n\n```\n$ sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi\n\nTue Apr 27 06:46:09 2021       \n+-----------------------------------------------------------------------------+\n| NVIDIA-SMI 450.102.04   Driver Version: 450.102.04   CUDA Version: 11.0     |\n|-------------------------------+----------------------+----------------------+\n| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n|                               |                      |               MIG M. |\n|===============================+======================+======================|\n|   0  GeForce GTX 1080    Off  | 00000000:01:00.0  On |                  N/A |\n| N/A   56C    P0    42W /  N/A |   1809MiB /  8114MiB |      0%      Default |\n|                               |                      |                  N/A |\n+-------------------------------+----------------------+----------------------+\n                                                                               \n+-----------------------------------------------------------------------------+\n| Processes:                                                                  |\n|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n|        ID   ID                                                   Usage      |\n|=============================================================================|\n+-----------------------------------------------------------------------------+\n\n```\nWe use environmental variables to specify the arguments.\nThe variables that can be specified and their meanings are as follows:\n\n| name | optional | default | description\n---- | ---- | ---- | ----\n| HOST_DATADIR    | False | | Path to the directory where the dataset is located in the host.\n| HOST_OUTPUTSDIR | False | | Path to the directory where the output will be located in the host.\n| HOST_WEIGHTDIR  | False | | Path to the directory where the pretrained wight is located in the host.\n| WEIGHTFILE      | False | | File name of the pretrained wight.\n| ARCH       | True | resnet56 | Name of the architecture.\n| BATCH_SIZE | True | 512      | Size of batch.\n| DATASET    | True | cifar10  | Name of dataset.\n| EPS        | True | 4.0      | L2 norm size of Fourier basis.\n| IGNORE_EDGE_SIZE  | True | 0 | Size of the edge to ignore.\n| NUM_SAMPLES       | True | -1| Number of samples used from dataset. If -1, use all samples.\n| NVIDIA_VISIBLE_DEVICES   | True | 0 | Device number (or list of number) visible from CUDA.\n\nFor example:\n\n```\n$ export HOST_DATADIR=[DATASET_DIRECTORY_PATH]\n$ export HOST_OUTPUTSDIR=[OUTPUTS_DIRECTORY_PATH]\n$ export HOST_WEIGHTDIR=[WEIGHT_DIRECTORY_PATH]\n$ export WEIGHTFILE=[PYTORCH_MODEL_FILE]\n$ cd provision/docker\n$ sudo -E docker-compose up  # -E option is needed to inherit environment variables.\n```\n\nwill generate 31x31 Fourier Heat Map for ResNet56 on CIFAR-10 dataset and save as an image under `OUTPUTS_DIRECTORY_PATH/eval_fhmap/`.\n\n## References\n\n- [Dong Yin, Raphael Gontijo Lopes, Jonathon Shlens, Ekin D. Cubuk, Justin Gilmer. \"A Fourier Perspective on Model Robustness in Computer Vision.\", in NeurIPS, 2019.](https://arxiv.org/abs/1906.08988)\n- [Justin Gilmer and Dan Hendrycks. \"Adversarial Example Researchers Need to Expand What is Meant by ‘Robustness’.\", in Distill, 2019.](https://distill.pub/2019/advex-bugs-discussion/response-1/)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgatheluck%2FFourierHeatmap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgatheluck%2FFourierHeatmap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgatheluck%2FFourierHeatmap/lists"}