{"id":13788346,"url":"https://github.com/gmh14/RobNets","last_synced_at":"2025-05-12T02:33:12.454Z","repository":{"id":112989681,"uuid":"245825023","full_name":"gmh14/RobNets","owner":"gmh14","description":"[CVPR 2020] When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks","archived":false,"fork":false,"pushed_at":"2020-10-21T02:52:21.000Z","size":332,"stargazers_count":123,"open_issues_count":10,"forks_count":15,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-11-18T02:37:21.901Z","etag":null,"topics":["adversarial-attacks","adversarial-examples","deep-learning-architectures","neural-architecture-search","robustness"],"latest_commit_sha":null,"homepage":"http://www.mit.edu/~yuzhe/robnets.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gmh14.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-03-08T14:01:51.000Z","updated_at":"2024-10-01T07:00:33.000Z","dependencies_parsed_at":"2024-01-07T03:51:31.670Z","dependency_job_id":"22a0409f-1732-4c08-a368-cf73160d6f68","html_url":"https://github.com/gmh14/RobNets","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gmh14%2FRobNets","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gmh14%2FRobNets/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gmh14%2FRobNets/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gmh14%2FRobNets/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gmh14","download_url":"https://codeload.github.com/gmh14/RobNets/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253662789,"owners_count":21944131,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-attacks","adversarial-examples","deep-learning-architectures","neural-architecture-search","robustness"],"created_at":"2024-08-03T21:00:43.945Z","updated_at":"2025-05-12T02:33:12.118Z","avatar_url":"https://github.com/gmh14.png","language":"Python","funding_links":[],"categories":["1.) Neural Architecture Search"],"sub_categories":["**[Papers]**"],"readme":"# When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks \nThis repository contains the implementation code for paper [When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks](https://arxiv.org/abs/1911.10695) (__CVPR 2020__). Also check out the [project page](http://www.mit.edu/~yuzhe/robnets.html).\n\nIn this work, we take an architectural perspective and investigate the patterns of network architectures that are resilient to adversarial attacks. We discover a family of robust architectures (__RobNets__), which exhibit superior robustness performance to other widely used architectures.\n\n![overview](assets/robnets.png)\n\n## Installation\n\n### Prerequisites\n- __Data:__ Download the [CIFAR10](https://www.cs.toronto.edu/~kriz/cifar.html), [SVHN](http://ufldl.stanford.edu/housenumbers/) and [ImageNet](http://image-net.org/download) dataset and move the test/validation set to the folder `data/`.\n\n- __Model:__ Download the [pre-trained models](https://drive.google.com/file/d/12Dkycpdyo7el-SmwImFzp0-NIhGt3-AT/view?usp=sharing) and unzip to the folder `checkpoint/`.\n\n\n### Dependencies for RobNets \nYou can install the dependencies for RobNets using\n```bash\npip install -r requirements.txt\n```\n\n\n## Experiments\n\n### Test\n\nAll the configurations of the experiments are provided in folders `experiments/*/config.py`, including different datasets and RobNet architectures. You can directly modify them to suit your demand.\n\nTo conduct a specific experiment, e.g. `RobNet_free` for CIFAR10, run\n```bash\npython main.py --config='./experiments/RobNet_free_cifar10/config.py' --eval_only\n```\nWith the flag `eval_only`, you can test the results for all the experiments in `experiments`.\n\n### Train (NEW)\n\nWe also provide the training interface of RobNets. For now, only training on CIFAR10 is provided. Training on ImageNet is WIP. \n\nWe use [Pytorch distributed training](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html) with [slurm](https://slurm.schedmd.com/tutorials.html) and [nccl](https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/overview.html) backend. You can conduct the training for `RobNet_large` on CIFAR10 by running\n```bash\nGPUS_PER_NODE=8 GPUS=32 bash slurm_train.sh **PartitionName** './experiments/RobNet_large_cifar10/config.py'\n```\n\n`RobNet_free_cifar10` and `RobNet_large_v1_cifar10` in `checkpoint/` are obtained with a total training batch size `1536`, while `RobNet_large_v2_cifar10` with batch size `1024`. Make sure to linearly scale the learning rate if you have a different batch size. (In fact, the hyper-parameters here are not optimized sufficiently by trial and error. If you find a better combination, welcome to deliver PR!)\n\nNote: You may notice that some of the training configurations are slightly different from the original paper, such as the learning rate scheduler. However, the training configurations in this repo can yeild even better results than those in the paper. Check the training log of `RobNet_large_v1_cifar10` [here](https://drive.google.com/file/d/11S1F2-B4Lm5Sc7uFBE4bh8-QItwE2Stf/view?usp=sharing) using the script in this repo. You can try a test using this checkpoint and will get ~83.5% clean accuracy and ~52.1% adversarial accuracy under PGD-20 attack!\n\n## Use RobNet Architectures\nTo use the searched RobNet models, for example, load `RobNet_free` on CIFAR10:\n```python\nimport models\nimport architecture_code\nimport utils\n\n# use RobNet architecture\nnet = models.robnet(architecture_code.robnet_free)\nnet = net.cuda()\n# load pre-trained model\nutils.load_state('./checkpoint/RobNet_free_cifar10.pth.tar', net)\n```\nFor other models, the loading process is similar, just copy the corresponding parameters (you can find in the variable `model_param` in each `experiments/*/config.py`) to the function `models.robnet()`.\n\n\n## Acknowledgements\nThe implementation of RobNets is partly based on [this work](https://github.com/quark0/darts).\n\n\n## Citation\nIf you find the idea or code useful for your research, please cite [our paper](https://arxiv.org/abs/1911.10695):\n```bib\n@article{guo2019meets,\n  title={When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks},\n  author={Guo, Minghao and Yang, Yuzhe and Xu, Rui and Liu, Ziwei and Lin, Dahua},\n  journal={arXiv preprint arXiv:1911.10695},\n  year={2019}\n}\n```\n\n\n## Contact\nPlease contact guomh2014@gmail.com and yuzheyangpku@gmail.com if you have any questions. Enjoy!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgmh14%2FRobNets","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgmh14%2FRobNets","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgmh14%2FRobNets/lists"}