{"id":13738172,"url":"https://github.com/alibaba/easyrobust","last_synced_at":"2025-10-14T08:46:17.546Z","repository":{"id":61107431,"uuid":"462110806","full_name":"alibaba/easyrobust","owner":"alibaba","description":"EasyRobust: an Easy-to-use library for state-of-the-art Robust Computer Vision Research with PyTorch.","archived":false,"fork":false,"pushed_at":"2024-06-30T13:01:46.000Z","size":19966,"stargazers_count":332,"open_issues_count":5,"forks_count":38,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-05-08T16:45:33.438Z","etag":null,"topics":["adversarial-robustness","deep-learning","image-classification","imagenet","pretrained-models","robustness"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alibaba.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-22T02:42:20.000Z","updated_at":"2025-04-27T05:45:33.000Z","dependencies_parsed_at":"2024-01-07T17:11:46.995Z","dependency_job_id":"1a28fd7b-83a2-4735-bd7a-068ff64962c4","html_url":"https://github.com/alibaba/easyrobust","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/alibaba/easyrobust","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alibaba%2Feasyrobust","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alibaba%2Feasyrobust/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alibaba%2Feasyrobust/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alibaba%2Feasyrobust/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alibaba","download_url":"https://codeload.github.com/alibaba/easyrobust/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alibaba%2Feasyrobust/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279018299,"owners_count":26086345,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-robustness","deep-learning","image-classification","imagenet","pretrained-models","robustness"],"created_at":"2024-08-03T03:02:13.245Z","updated_at":"2025-10-14T08:46:12.529Z","avatar_url":"https://github.com/alibaba.png","language":"Jupyter Notebook","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# EasyRobust\n\n\u003cdiv align=\"center\"\u003e\n\n[![license](https://img.shields.io/github/license/alibaba/easyrobust.svg)](https://github.com/alibaba/easyrobust/blob/main/LICENSE)\n[![open issues](https://isitmaintained.com/badge/open/alibaba/easyrobust.svg)](https://github.com/alibaba/easyrobust/issues)\n[![GitHub pull-requests](https://img.shields.io/github/issues-pr/alibaba/easyrobust.svg)](https://GitHub.com/alibaba/easyrobust/pull/)\n[![GitHub latest release](https://badgen.net/github/release/alibaba/easyrobust)](https://GitHub.com/alibaba/easyrobust/releases/)\n\u003c/div\u003e\n\n## What's New\n- **[Apr 2024]** [Revisiting and Exploring Efficient Fast Adversarial Training via LAW: Lipschitz Regularization and Auto Weight Averaging](https://arxiv.org/abs/2308.11443) was accepted by T-IFS 2024! Codes will be avaliable at [examples/imageclassification/cifar10/adversarial_training/fgsm_law](examples/imageclassification/cifar10/adversarial_training/fgsm_law)\n\n- **[Jul 2023]** [COCO-O: A Benchmark for Object Detectors under Natural Distribution Shifts](https://arxiv.org/abs/2307.12730) was accepted by ICCV 2023! Dataset will be avaliable at [benchmarks/coco_o](benchmarks/coco_o)\n\n- **[Jul 2023]** [Robust Automatic Speech Recognition via WavAugment Guided Phoneme Adversarial Training](https://arxiv.org/abs/2307.12498) was accepted by INTERSPEECH 2023! Codes will be avaliable at [examples/asr/WAPAT](examples/asr/WAPAT)\n\n- **[Feb 2023]** [ImageNet-E: Benchmarking Neural Network Robustness against Attribute Editing](https://arxiv.org/abs/2303.17096) was accepted by CVPR 2023! Codes will be avaliable at [benchmarks/imagenet-e](benchmarks/imagenet-e) \n\n- **[Feb 2023]** [TransAudio: Towards the Transferable Adversarial Audio Attack via Learning Contextualized Perturbations](https://arxiv.org/abs/2303.15940) was accepted by ICASSP 2023! Codes will be avaliable at [examples/attacks/transaudio](examples/attacks/transaudio) \n\n- **[Jan 2023]** [Inequality phenomenon in $l_\\infty$-adversarial training, and its unrealized threats](https://openreview.net/pdf?id=4t9q35BxGr) was accepted by ICLR 2023 as **notable-top-25%**! Codes will be avaliable at [examples/attacks/inequality](examples/attacks/inequality)\n\n- **[Oct 2022]**: [Towards Understanding and Boosting Adversarial Transferability from a Distribution Perspective](https://arxiv.org/abs/2210.04213) was accepted by TIP 2022! Codes will be avaliable at [examples/attacks/dra](examples/attacks/dra)\n\n- **[Sep 2022]**: [Boosting Out-of-distribution Detection with Typical Features](https://arxiv.org/abs/2210.04200) was accepted by NeurIPS 2022! Codes avaliable at [examples/ood_detection/BATS](examples/ood_detection/BATS)\n\n- **[Sep 2022]**: [Enhance the Visual Representation via Discrete Adversarial Training](https://arxiv.org/abs/2209.07735) was accepted by NeurIPS 2022! Codes avaliable at [examples/imageclassification/imagenet/dat](examples/imageclassification/imagenet/dat)\n\n- **[Sep 2022]**: Updating 5 methods for analysing your robust models under [tools/](tools).\n\n- **[Sep 2022]**: Updating 13 reproducing examples of robust training methods under [examples/imageclassification/imagenet](examples/imageclassification/imagenet).\n\n- **[Sep 2022]**: Releasing 16 Adversarial Training models, including a Swin-B which achieves SOTA adversairal robustness with 47.42% on AutoAttack!\n\n- **[Sep 2022]**: EasyRobust v0.2.0 released.\n\n## Our Research Project\n\n- **[T-IFS 2024]** Revisiting and Exploring Efficient Fast Adversarial Training via LAW: Lipschitz Regularization and Auto Weight Averaging [[Paper](https://arxiv.org/abs/2308.11443), [Code](examples/imageclassification/cifar10/adversarial_training/fgsm_law)]\n- **[ICCV 2023]** COCO-O: A Benchmark for Object Detectors under Natural Distribution Shifts [[Paper](https://arxiv.org/abs/2307.12730), [COCO-O dataset](benchmarks/coco_o)]\n- **[INTERSPEECH 2023]** Robust Automatic Speech Recognition via WavAugment Guided Phoneme Adversarial Training [[Paper](https://arxiv.org/abs/2307.12498), [Code](examples/asr/WAPAT)]\n- **[CVPR 2023]** ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editing [[Paper](https://arxiv.org/abs/2303.17096), [Image editing toolkit](benchmarks/imagenet-e/ImageNet-Editing), [ImageNet-E dataset](https://drive.google.com/file/d/19M1FQB8c_Mir6ermRsukTQReI-IFXeT0/view?usp=sharing)]\n- **[ICLR 2023]** Inequality phenomenon in $l_\\infty$-adversarial training, and its unrealized threats [[Paper](https://openreview.net/pdf?id=4t9q35BxGr), [Code](examples/attacks/inequality)]\n- **[ICASSP 2023]** TransAudio: Towards the Transferable Adversarial Audio Attack via Learning Contextualized Perturbations [[Paper](https://arxiv.org/abs/2303.15940), [Code](examples/attacks/transaudio)]\n- **[TIP 2022]** Towards Understanding and Boosting Adversarial Transferability from a Distribution Perspective [[Paper](https://arxiv.org/abs/2210.04213), [Code](examples/attacks/dra)]\n- **[NeurIPS 2022]** Boosting Out-of-distribution Detection with Typical Features [[Paper](https://arxiv.org/abs/2210.04200), [Code](examples/ood_detection/BATS)]\n- **[NeurIPS 2022]** Enhance the Visual Representation via Discrete Adversarial Training [[Paper](https://arxiv.org/abs/2209.07735), [Code](examples/imageclassification/imagenet/dat)]\n- **[CVPR 2022]** Towards Robust Vision Transformer [[Paper](https://arxiv.org/abs/2105.07926), [Code](examples/imageclassification/imagenet/rvt)]\n\n\n## Introduction\n\nEasyRobust is an **Easy**-to-use library for state-of-the-art **Robust** Computer Vision Research with [PyTorch](https://pytorch.org). EasyRobust aims to accelerate research cycle in robust vision, by collecting comprehensive robust training techniques and benchmarking them with various robustness metrics. The key features includes:\n\n- **Reproducible implementation of SOTA in Robust Image Classification**: Most existing SOTA in Robust Image Classification are implemented - [Adversarial Training](https://arxiv.org/abs/1706.06083), [AdvProp](https://arxiv.org/abs/1911.09665), [SIN](https://arxiv.org/abs/1811.12231), [AugMix](https://arxiv.org/abs/1912.02781), [DeepAugment](https://arxiv.org/abs/2006.16241), [DrViT](https://arxiv.org/abs/2111.10493), [RVT](https://arxiv.org/abs/2105.07926), [FAN](https://arxiv.org/abs/2204.12451), [APR](https://arxiv.org/abs/2108.08487), [HAT](https://arxiv.org/abs/2204.00993), [PRIME](https://arxiv.org/abs/2112.13547), [DAT](https://arxiv.org/abs/2209.07735) and so on.\n\n- **Benchmark suite**: Variety of benchmarks tasks including [ImageNet-A](https://arxiv.org/abs/1907.07174), [ImageNet-R](https://arxiv.org/abs/2006.16241), [ImageNet-Sketch](https://arxiv.org/abs/1905.13549), [ImageNet-C](https://arxiv.org/abs/1903.12261), [ImageNetV2](https://arxiv.org/abs/1902.10811), [Stylized-ImageNet](https://arxiv.org/abs/1811.12231), [ObjectNet](https://objectnet.dev/). \n\n- **Scalability**: You can use EasyRobust to conduct 1-gpu training, multi-gpu training on single machine and large-scale multi-node training.\n\n- **Model Zoo**: Open source more than 30 pretrained adversarially or non-adversarially robust models. \n\n- **Analytical tools**: Support analysis and visualization about a pretrained robust model, including [Attention Visualization](./tools), [Decision Boundary Visualization](./tools), [Convolution Kernel Visualization](./tools), [Shape vs. Texture Biases Analysis](./tools), etc. Using these tools can help us to explain how robust training improves the interpretability of the model. \n\n## Technical Articles\nWe have a series of technical articles on the functionalities of EasyRobust.\n - [NeurIPS2022 阿里浙大提出利用更典型的特征来提升分布外检测性能](https://mp.weixin.qq.com/s/EWBajwq2Yg_z92uw7wDtXQ)\n - [顶刊TIP 2022！阿里提出：从分布视角出发理解和提升对抗样本的迁移性](https://mp.weixin.qq.com/s/qtTXn3B4OYiBaZgHZo9cGA)\n - [无惧对抗和扰动、增强泛化，阿里安全打造更鲁棒的ViT模型，论文入选CVPR 2022](https://mp.weixin.qq.com/s/J6gqA09MxLwmN_C40Sjf1Q)\n - [NeurIPS2022 阿里提出基于离散化对抗训练的鲁棒视觉新基准](https://mp.weixin.qq.com/s?__biz=MzU1NTMyOTI4Mw==\u0026mid=2247610520\u0026idx=1\u0026sn=d7ff15f6a89030a01ca03de406f2f4ec)\n \n## Installation\n### Install from Source:\n```bash\n$ git clone https://github.com/alibaba/easyrobust.git\n$ cd easyrobust\n$ pip install -e .\n```\n\n### Install from PyPI:\n```bash\n$ pip install easyrobust\n```\n\ndownload the ImageNet dataset and place into `/path/to/imagenet`. Specify `$ImageNetDataDir` as ImageNet path by:\n\n```bash\n$ export ImageNetDataDir=/path/to/imagenet\n```\n\n**[Optional]:** If you use EasyRobust to evaluate the model robustness, download the benchmark dataset by:\n```bash\n$ sh download_data.sh\n```\n\n**[Optional]:** If you use analysis tools in `tools/`, install extra requirements by:\n```bash\n$ pip install -r requirements/optional.txt\n```\n\n\n### Docker\nWe have provided a runnable environment in `docker/Dockerfile` for users who do not want to install by pip. To use it, please confirm that `docker` and `nvidia-docker` have installed. Then run the following command:\n```bash\ndocker build -t alibaba/easyrobust:v1 -f docker/Dockerfile . \n```\n\n## Getting Started\nEasyRobust focuses on the basic usages of: **(1) Evaluate and benchmark the robustness of a pretrained models** and **(2) Train your own robust models or reproduce the results of previous SOTA methods**.\n\n### 1. How to evaluate and benchmark the robustness of given models?\nIt only requires a few lines to evaluate the robustness of a model using EasyRobust. We give a minimalist example in [benchmarks/resnet50_example.py](./benchmarks/resnet50_example.py):\n\n```python\n#############################################################\n#         Define your model\n#############################################################\nmodel = torchvision.models.resnet50(pretrained=True)\nmodel = model.eval()\nif torch.cuda.is_available(): model = model.cuda()\n\n#############################################################\n#         Start Evaluation\n#############################################################\n\n# ood\nevaluate_imagenet_val(model, 'benchmarks/data/imagenet-val')\nevaluate_imagenet_a(model, 'benchmarks/data/imagenet-a')\nevaluate_imagenet_r(model, 'benchmarks/data/imagenet-r')\nevaluate_imagenet_sketch(model, 'benchmarks/data/imagenet-sketch')\nevaluate_imagenet_v2(model, 'benchmarks/data/imagenetv2')\nevaluate_stylized_imagenet(model, 'benchmarks/data/imagenet-style')\nevaluate_imagenet_c(model, 'benchmarks/data/imagenet-c')\n# objectnet is optional since it spends a lot of disk storage. we skip it here. \n# evaluate_objectnet(model, 'benchmarks/data/ObjectNet/images')\n\n# adversarial\nevaluate_imagenet_autoattack(model, 'benchmarks/data/imagenet-val')\n```\nYou can do evaluation by simply running the command: `python benchmarks/resnet50_example.py`. After running is completed, your will get the following output:\n```\nTop1 Accuracy on the ImageNet-Val: 76.1%\nTop1 Accuracy on the ImageNet-A: 0.0%\nTop1 Accuracy on the ImageNet-R: 36.2%\nTop1 Accuracy on the ImageNet-Sketch: 24.1%\nTop1 Accuracy on the ImageNet-V2: 63.2%\nTop1 Accuracy on the Stylized-ImageNet: 7.4%\nTop1 accuracy 39.2%, mCE: 76.7 on the ImageNet-C\nTop1 Accuracy on the AutoAttack: 0.0%\n```\n\n### 2. How to use EasyRobust to train my own robust models?\nWe implement most robust training methods in the folder `examples/imageclassification/imagenet/`. All of them are based on a basic training script: [examples/imageclassification/imagenet/base_training_script.py](./examples/imageclassification/imagenet/base_training_script.py). By comparing the difference, you can clearly see where and which hyperparameters of basic training are modified to create a robust training example. Below we present the tutorials of some classic methods:\n- [Adversarial Training on ImageNet using 8 GPUs](./examples/imageclassification/imagenet/adversarial_training)\n- [AugMix Training on ImageNet with 180 Epochs](./examples/imageclassification/imagenet/augmix)\n- [AdvProp for Improving Non-adversarial Robustness and Accuracy](./examples/imageclassification/imagenet/advprop)\n- [Using Stylized ImageNet as Extended Data for Training](./examples/imageclassification/imagenet/SIN)\n- [Discrete Adversarial Training for ViTs](./examples/imageclassification/imagenet/dat)\n- [Training Robust Vision Transformers (RVT) with 300 Epochs](./examples/imageclassification/imagenet/rvt)\n- [Robust Finetuning of CLIP Models](./examples/imageclassification/imagenet/wiseft)\n\n## Analytical Tools\n\nsee [tools/README.md](./tools)\n\n## Model Zoo and Baselines\n\n\n### Submit your models\nWe provide a tool `benchmarks/benchmark.py` to help users directly benchmark their models:\n```\nUsage: \n    python benchmarks/benchmark.py [OPTIONS...]\n\nOPTIONS:\n    --model [ARCH in timm]\n    --data_dir [PATH of the bencmark datasets]\n    --ckpt_path [URL or PATH of the model weights]\n```\nIf you are willing to submit the model to our benchmarks, you can prepare a python script similar to `benchmarks/benchmark.py` and weights file `xxx.pth`, zip all the files. Then open an issue with the \"Submit Model\" template and provide a json storing submit information. Below is a submission template in adversarial robustness benchmark of image classification:\n\n```markdown\n## Submit Json Information\n\n{\"date\": \"19/06/2017\", \n \"extra_data\": \"no\", \n \"model\": \"\u003cb\u003eAdversarial Training\u003c/b\u003e\", \n \"institution\": \"MIT\", \n \"paper_link\": \"https://arxiv.org/abs/1706.06083\", \n \"code_link\": \"\", \n \"architecture\": \"swin-b\", \n \"training framework\": \"easyrobust (v1)\", \n \"ImageNet-val\": 75.05, \n \"autoattack\": 47.42, \n \"files\": \"\u003ca href=http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_swin_base_patch4_window7_224_ep4.pth \u003edownload\u003c/a\u003e\", \n \"advrob_imgcls_leaderboard\": true, \n \"oodrob_imgcls_leaderboard\": false, \n \"advrob_objdet_leaderboard\": false, \n \"oodrob_objdet_leaderboard\": false}\n```\nWe will check the result and present your result into the benchmark if there is no problem. For submission template of other benchmarks, check [submit-model.md](.github/ISSUE_TEMPLATE/submit-model.md).\n\nBelow is the model zoo and benchmark of the EasyRobust. All the results are runned by [benchmarks/adv_robust_bench.sh](./benchmarks/adv_robust_bench.sh) and [benchmarks/non_adv_robust_bench.sh](./benchmarks/non_adv_robust_bench.sh).\n\n### Adversarial Robust Benchmark (sorted by AutoAttack)\n\n| Training Framework | Method | Model | ImageNet-Val | AutoAttack | Files |\n| ---- | :----: | :----: | :----: | :----: | :----: |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [Swin-B](https://arxiv.org/abs/2103.14030) | 75.05% | 47.42% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_swin_base_patch4_window7_224_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [Swin-S](https://arxiv.org/abs/2103.14030) | 73.41% | 46.76% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_swin_small_patch4_window7_224_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [ViT-B/16](https://arxiv.org/abs/2010.11929) | 70.64% | 43.04% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_vit_base_patch16_224_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [EfficientNet-B3](https://arxiv.org/abs/1905.11946) | 67.65% | 41.72% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_efficientnet_b3_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [ResNet101](](https://arxiv.org/abs/1512.03385)) | 69.51% | 41.04% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_resnet101_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [ViT-S/16](https://arxiv.org/abs/2010.11929) | 66.43% | 39.20% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_vit_small_patch16_224_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [EfficientNet-B2](https://arxiv.org/abs/1905.11946) | 64.75% | 38.54% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_efficientnet_b2_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [ResNeSt50d](https://arxiv.org/abs/2004.08955) | 70.03% | 38.52% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_resnest50d_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [ViT-B/32](https://arxiv.org/abs/2010.11929) | 65.58% | 37.38% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_vit_base_patch32_224_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [EfficientNet-B1](https://arxiv.org/abs/1905.11946) | 63.99% | 37.20% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_efficientnet_b1_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [SEResNet101](https://arxiv.org/abs/1709.01507) | 71.11% | 37.18% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_seresnet101_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [ResNeXt50_32x4d](https://arxiv.org/abs/1611.05431) | 67.39% | 36.42% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_resnext50_32x4d_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [EfficientNet-B0](https://arxiv.org/abs/1905.11946) | 61.83% | 35.06% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_efficientnet_b0_ep4.pth) |\n| [robustness](https://github.com/MadryLab/robustness) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [ResNet50](https://arxiv.org/abs/1512.03385) | 64.02% | 34.96% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/robustbench/robustness_advtrain_resnet50_linf_eps4.0.pth) |\n| **EasyRobust (Ours)** | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [ResNet50](https://arxiv.org/abs/1512.03385) | 65.1% | 34.9% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/adversarial_training/model_best.pth.tar)/[args](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/adversarial_training/args.yaml)/[logs](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/adversarial_training/summary.csv) | \n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [SEResNet50](https://arxiv.org/abs/1709.01507) | 66.68% | 33.56% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_seresnet50_ep4.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [DenseNet121](https://arxiv.org/abs/1608.06993) | 60.90% | 29.78% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_densenet121_ep4.pth) |\n| [Official](https://github.com/mahyarnajibi/FreeAdversarialTraining) | [Free AT](https://arxiv.org/abs/1904.12843) | [ResNet50](https://arxiv.org/abs/1512.03385) | 59.96% | 28.58% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/robustbench/free_adv_step4_eps4_repeat4.pth) |\n| [Official](https://github.com/locuslab/fast_adversarial) | [FGSM AT](https://arxiv.org/abs/2001.03994) | [ResNet50](https://arxiv.org/abs/1512.03385) | 55.62% | 26.24% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/robustbench/fastadv_imagenet_model_weights_4px.pth) |\n| EasyRobust (V1) | [Adversarial Training](https://arxiv.org/abs/1706.06083) | [VGG16](https://arxiv.org/abs/1409.1556) | 59.96% | 25.92% | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/imagenet_pretrained_models/advtrain_models/advtrain_vgg16_ep4.pth) |\n\n### Non-Adversarial Robust Benchmark (sorted by ImageNet-C)\n| Training Framework | Method | Model | Files | ImageNet-Val | V2 | C (mCE↓) | R | A | Sketch| Stylized | ObjectNet |\n| ---- | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: |\n| **EasyRobust (Ours)** | [DAT](http://arxiv.org/abs/2209.07735) | [ViT-B/16](https://arxiv.org/abs/2010.11929) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/dat/model_best.pth.tar)/[args](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/dat/args.yaml)/[logs](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/dat/summary.csv) | 81.38% | 69.99% | 45.59 | 49.64% | 24.61% | 36.46% | 24.84% | 20.12% |\n| **EasyRobust (Ours)** | - | [RVT-S*](https://arxiv.org/abs/2105.07926) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/rvt/model_best.pth.tar)/[args](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/rvt/args.yaml)/[logs](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/rvt/summary.csv) | 82.10% | 71.40% | 48.22 | 47.84% | 26.93% | 35.34% | 20.71% | 23.24% |\n| [Official](https://github.com/vtddggg/Robust-Vision-Transformer) | - | [RVT-S*](https://arxiv.org/abs/2105.07926) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/official_models/rvt_small_plus.pth) | 81.82% | 71.05% | 49.42 | 47.33% | 26.53% | 34.22% | 20.48% | 23.11% |\n| **EasyRobust (Ours)** | - | [DrViT-S](https://arxiv.org/abs/2111.10493) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/drvit/model_best.pth.tar)/[args](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/drvit/args.yaml)/[logs](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/drvit/summary.csv) | 80.66% | 69.62% | 49.96 | 43.68% | 20.79% | 31.13% | 17.89% | 20.50% |\n| - | - | [DrViT-S](https://arxiv.org/abs/2111.10493) | - | 77.03% | 64.49% | 56.89 | 39.02% | 11.85% | 28.78% | 14.22% | 26.49% |\n| [Official](https://github.com/amodas/PRIME-augmentations) | [PRIME](https://arxiv.org/abs/2112.13547) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/official_models/prime.pth) | 76.91% | 65.42% | 57.49 | 42.20% | 2.21% | 29.82% | 13.94% | 16.59% |\n| **EasyRobust (Ours)** | [PRIME](https://arxiv.org/abs/2112.13547) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/prime/model_best.pth.tar)/[args](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/prime/args.yaml)/[logs](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/prime/summary.csv) | 76.64% | 64.37% | 57.62 | 41.95% | 2.07% | 29.63% | 13.56% | 16.28% |\n| **EasyRobust (Ours)** | [DeepAugment](https://arxiv.org/abs/2006.16241) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/deepaugment/model_best.pth.tar)/[args](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/deepaugment/args.yaml)/[logs](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/deepaugment/summary.csv) | 76.58% | 64.77% | 60.27 | 42.80% | 3.62% | 29.65% | 14.88% | 16.88% |\n| [Official](https://github.com/hendrycks/imagenet-r) | [DeepAugment](https://arxiv.org/abs/2006.16241) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/official_models/deepaugment.pth) | 76.66% | 65.24% | 60.37 | 42.17% | 3.46% | 29.50% | 14.68% | 17.13% |\n| **EasyRobust (Ours)** | [Augmix](https://arxiv.org/abs/1912.02781) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/augmix/model_best.pth.tar)/[args](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/augmix/args.yaml)/[logs](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/augmix/summary.csv) | 77.81% | 65.60% | 64.14 | 43.34% | 4.04% | 29.81% | 12.33% | 17.21% |\n| **EasyRobust (Ours)** | [APR](https://arxiv.org/abs/2108.08487) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/apr/model_best.pth.tar)/[args](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/apr/args.yaml)/[logs](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/apr/summary.csv) | 76.28% | 64.78% | 64.89 | 42.17% | 4.18% | 28.90% | 13.03% | 16.78% |\n| [Official](https://github.com/google-research/augmix) | [Augmix](https://arxiv.org/abs/1912.02781) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/official_models/augmix.pth) | 77.54% | 65.42% | 65.27 | 41.04% | 3.78% | 28.48% | 11.24% | 17.54% |\n| [Official](https://github.com/iCGY96/APR) | [APR](https://arxiv.org/abs/2108.08487) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/official_models/apr_sp.pth) | 75.61% | 64.24% | 65.56 | 41.35% | 3.20% | 28.37% | 13.01% | 16.61% |\n| [Official](https://github.com/LiYingwei/ShapeTextureDebiasedTraining) | [S\u0026T Debiased](https://arxiv.org/abs/2010.05981) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/official_models/debiased.pth) | 76.91% | 65.04% | 67.55 | 40.81% | 3.50% | 28.41% | 17.40% | 17.38% |\n| **EasyRobust (Ours)** | [SIN+IN](https://arxiv.org/abs/1811.12231) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/sin/model_best.pth.tar)/[args](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/sin/args.yaml)/[logs](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/sin/summary.csv) | 75.46% | 63.50% | 67.73 | 42.34% | 2.47% | 31.39% | 59.37% | 16.17% |\n| [Official](https://github.com/rgeirhos/texture-vs-shape) | [SIN+IN](https://arxiv.org/abs/1811.12231) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/official_models/sin_in.pth) | 74.59% | 62.43% | 69.32 | 41.45% | 1.95% | 29.69% | 57.38% | 15.93% |\n| [Non-Official](https://github.com/tingxueronghua/pytorch-classification-advprop) | [AdvProp](https://arxiv.org/abs/1911.09665) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/official_models/advprop.pth) | 77.04% | 65.27% | 70.81 | 40.13% | 3.45% | 25.95% | 10.01% | 18.23% |\n| **EasyRobust (Ours)** | [S\u0026T Debiased](https://arxiv.org/abs/2010.05981) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/shape_texture_debiased_training/model_best.pth.tar)/[args](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/shape_texture_debiased_training/args.yaml)/[logs](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/shape_texture_debiased_training/summary.csv) | 77.21% | 65.10% | 70.98 | 38.59% | 3.28% | 26.09% | 14.59% | 16.99% |\n| **EasyRobust (Ours)** | [AdvProp](https://arxiv.org/abs/1911.09665) | [ResNet50](https://arxiv.org/abs/1512.03385) | [ckpt](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/advprop/model_best.pth.tar)/[args](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/advprop/args.yaml)/[logs](http://alisec-competition.oss-cn-shanghai.aliyuncs.com/xiaofeng/easy_robust/benchmark_models/ours/examples/advprop/summary.csv) | 76.64% | 64.35% | 77.64 | 37.43% | 2.83% | 24.71% | 7.33% | 16.82% |\n\n## Credits\nEasyRobust concretizes previous excellent works by many different authors. We'd like to thank, in particular, the following implementations which have helped us in our development:\n- [timm](https://github.com/rwightman/pytorch-image-models) @rwightman and the training script.\n- [robustness](https://github.com/MadryLab/robustness) @MadryLab and [autoattack](https://github.com/fra31/auto-attack) @fra31 for attack implementation. \n- [modelvshuman](https://github.com/bethgelab/model-vs-human) @bethgelab for model analysis.\n- [AdaIN](https://github.com/naoto0804/pytorch-AdaIN) @naoto0804 for style trnsfer and [VQGAN](https://github.com/CompVis/taming-transformers) @CompVis for image discretization. \n- All the authors and implementations of the robustness research work we refer in this library.  \n\n## Citing EasyRobust\n\nWe provide a BibTeX entry for users who apply EasyRobust to help their research: \n\n```BibTeX\n@misc{mao2022easyrobust,\n  author =       {Xiaofeng Mao and Yuefeng Chen and Xiaodan Li and Gege Qi and Ranjie Duan and Rong Zhang and Hui Xue},\n  title =        {EasyRobust: A Comprehensive and Easy-to-use Toolkit for Robust Computer Vision},\n  howpublished = {\\url{https://github.com/alibaba/easyrobust}},\n  year =         {2022}\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falibaba%2Feasyrobust","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falibaba%2Feasyrobust","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falibaba%2Feasyrobust/lists"}