Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tootouch/sid
pytorch reimplementation for Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain
https://github.com/tootouch/sid
Last synced: 2 days ago
JSON representation
pytorch reimplementation for Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain
- Host: GitHub
- URL: https://github.com/tootouch/sid
- Owner: TooTouch
- License: apache-2.0
- Created: 2022-07-24T16:39:51.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2022-10-30T06:39:24.000Z (about 2 years ago)
- Last Synced: 2024-03-15T19:22:30.058Z (8 months ago)
- Language: Python
- Size: 249 KB
- Stars: 10
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain
Pytorch re-implementation for "Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain". AAAI 2021
- **paper**: [Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain](https://arxiv.org/abs/2103.04302)
- **official code**: https://github.com/JinyuTian/SID
# Run
run [`./scripts/run_pipeline.sh`](https://github.com/TooTouch/SID/blob/main/scripts/run_pipeline.sh)
**run_pipeline.sh**
```bash
modelname_list="vgg19 resnet34"
adv_method_list=("DeepFool" "BIM" "CW" "CW" "FAB" "FGSM" "PGD" "PGD" "PGD")
adv_expname_list=("DeepFool" "BIM" "CW" "Low_CW" "FAB" "FGSM" "PGD" "Low_PGD1" "Low_PGD2")
dataname_list="CIFAR10 SVHN CIFAR100"
for modelname in $modelname_list
do
for dataname in $dataname_list
do
# 1. train classifier
bash run_classifier.sh $modelname $dataname
# 2. make adversarial examples
for i in ${!adv_method_list[*]}
do
bash save_adv_samples.sh $modelname ${adv_method_list[$i]} ${adv_expname_list[$i]} $dataname
done
# 3. known attack
for i in ${!adv_method_list[*]}
do
bash known_attack.sh $modelname ${adv_expname_list[$i]} $dataname
done
# 4. transfer attack
bash run_transfer_attack.sh $modelname $dataname
done
done
```
# Results
- **Model**: ResNet34, VGG19
## 1. Adversarial Attacks
**CIFAR10**
- VGG19
| | Adv Acc(%) | Adv Acc(%) DWT | # Success Images |
|:-----------|-------------:|-----------------:|-------------------:|
| DeepFool | 1.3 | 84.73 | 7117 |
| BIM | 0 | 63.03 | 7213 |
| CW | 12.44 | 80.84 | 5993 |
| Low_CW | 52.96 | 87.62 | 2399 |
| FAB | 0.03 | 87.52 | 7240 |
| FGSM | 13.82 | 59.65 | 5872 |
| PGD | 0 | 64.92 | 7204 |
| PGD_L2 | 0 | 65.4 | 7143 |
| Low_PGD1 | 59.34 | 86.9 | 1879 |
| Low_PGD2 | 15.96 | 84.4 | 5618 |
| AutoAttack | 0 | 68.12 | 7256 |
| Square | 0.81 | 81.66 | 7164 |
- ResNet34
| | Adv Acc(%) | Adv Acc(%) DWT | # Success Images |
|:-----------|-------------:|-----------------:|-------------------:|
| DeepFool | 5.59 | 90.66 | 6576 |
| BIM | 0.08 | 63.94 | 7022 |
| CW | 22.66 | 83.57 | 4852 |
| Low_CW | 58.79 | 91.86 | 1973 |
| FAB | 0 | 92.13 | 7021 |
| FGSM | 35.8 | 66.79 | 3825 |
| PGD | 0.06 | 67.95 | 7009 |
| PGD_L2 | 0.43 | 66.59 | 6991 |
| Low_PGD1 | 60.67 | 90.9 | 1704 |
| Low_PGD2 | 14.4 | 88.23 | 5606 |
| AutoAttack | 0 | 70.23 | 7070 |
| Square | 0.88 | 86.27 | 6940 |
**CIFAR100**
- VGG19
| | Adv Acc(%) | Adv Acc(%) DWT | # Success Images |
|:-----------|-------------:|-----------------:|-------------------:|
| DeepFool | 2.9 | 57.6 | 2961 |
| BIM | 1.72 | 44.59 | 2950 |
| CW | 9.35 | 52.97 | 2245 |
| Low_CW | 30.82 | 59.3 | 1187 |
| FAB | 5.31 | 59.14 | 2865 |
| FGSM | 16.79 | 35.4 | 1826 |
| PGD | 1.32 | 45.68 | 2943 |
| PGD_L2 | 2.07 | 47.33 | 2895 |
| Low_PGD1 | 29.61 | 58.47 | 1190 |
| Low_PGD2 | 9.72 | 56.72 | 2325 |
| AutoAttack | 0 | 47.42 | 3117 |
| Square | 2.45 | 51.7 | 2850 |
- ResNet34
| | Adv Acc(%) | Adv Acc(%) DWT | # Success Images |
|:-----------|-------------:|-----------------:|-------------------:|
| DeepFool | 11.49 | 69.42 | 3257 |
| BIM | 0.07 | 41.93 | 3811 |
| CW | 9.81 | 59.15 | 2890 |
| Low_CW | 31.38 | 68.75 | 1549 |
| FAB | 3.54 | 69.66 | 3587 |
| FGSM | 13.16 | 35.23 | 2610 |
| PGD | 0.08 | 44.93 | 3767 |
| PGD_L2 | 0.23 | 46.04 | 3768 |
| Low_PGD1 | 35.93 | 67.45 | 1202 |
| Low_PGD2 | 8.45 | 63.67 | 2969 |
| AutoAttack | 0 | 49.85 | 3765 |
| Square | 0.45 | 60.86 | 3731 |
**SVHN**
- VGG19
| | Adv Acc(%) | Adv Acc(%) DWT | # Success Images |
|:-----------|-------------:|-----------------:|-------------------:|
| DeepFool | 1.4 | 54.58 | 23937 |
| BIM | 1.05 | 29.9 | 24015 |
| CW | 15.68 | 69.78 | 20208 |
| Low_CW | 74.88 | 92.11 | 4829 |
| FAB | 0.74 | 90.69 | 24116 |
| FGSM | 20.19 | 44.18 | 19039 |
| PGD | 1.26 | 31.61 | 23982 |
| PGD_L2 | 1.11 | 24.12 | 24024 |
| Low_PGD1 | 81.78 | 91.85 | 3085 |
| Low_PGD2 | 52.86 | 83.1 | 10551 |
| AutoAttack | 0.44 | 34.92 | 24170 |
| Square | 2.98 | 82.6 | 23531 |
- ResNet34
| | Adv Acc(%) | Adv Acc(%) DWT | # Success Images |
|:-----------|-------------:|-----------------:|-------------------:|
| DeepFool | 4.28 | 78.8 | 23173 |
| BIM | 2.42 | 31.17 | 23615 |
| CW | 33.62 | 74.71 | 15495 |
| Low_CW | 76.69 | 93.47 | 4394 |
| FAB | 1.24 | 93.41 | 23938 |
| FGSM | 40.35 | 55.49 | 13760 |
| PGD | 2.74 | 33.89 | 23566 |
| PGD_L2 | 2.74 | 24.23 | 23557 |
| Low_PGD1 | 81.72 | 92.56 | 3142 |
| Low_PGD2 | 49.66 | 83.99 | 11355 |
| AutoAttack | 0.43 | 38.36 | 24129 |
| Square | 4.81 | 81.37 | 23008 |
## 2. Known Attacks
**CIFAR10**
- VGG19
| | AUROC(%) | Detection Acc(%) | #(train, dev, test) |
|:-----------|-----------:|-------------------:|:----------------------|
| DeepFool | 91.76 | 86.14 | (4528, 754, 2265) |
| BIM | 99.79 | 98.27 | (5245, 874, 2624) |
| CW | 88.31 | 81.62 | (3896, 648, 1952) |
| Low_CW | 90.99 | 85.94 | (1574, 261, 790) |
| FAB | 97.3 | 92.56 | (4629, 770, 2317) |
| FGSM | 90.67 | 83.96 | (4207, 700, 2108) |
| PGD | 99.67 | 98 | (5161, 859, 2585) |
| PGD_L2 | 99.74 | 97.76 | (5157, 858, 2582) |
| Low_PGD1 | 80.74 | 74.68 | (1252, 208, 630) |
| Low_PGD2 | 89.74 | 85.5 | (3596, 598, 1801) |
| AutoAttack | 99.64 | 97.86 | (5102, 849, 2554) |
| Square | 97.59 | 94.23 | (4607, 766, 2306) |
- ResNet34
| | AUROC(%) | Detection Acc(%) | #(train, dev, test) |
|:-----------|-----------:|-------------------:|:----------------------|
| DeepFool | 92.94 | 89.1 | (4112, 684, 2059) |
| BIM | 99.39 | 97.18 | (5085, 846, 2546) |
| CW | 90.28 | 84.19 | (3184, 530, 1596) |
| Low_CW | 82.58 | 79.17 | (1267, 210, 637) |
| FAB | 95.76 | 92.1 | (4399, 732, 2204) |
| FGSM | 94.89 | 88.94 | (2863, 476, 1433) |
| PGD | 99.28 | 96.39 | (4935, 821, 2472) |
| PGD_L2 | 98.88 | 96 | (5022, 836, 2514) |
| Low_PGD1 | 82.95 | 78.13 | (1102, 183, 553) |
| Low_PGD2 | 90.07 | 85.16 | (3519, 586, 1761) |
| AutoAttack | 97.57 | 93.69 | (4919, 818, 2462) |
| Square | 95.32 | 91.77 | (4374, 728, 2191) |
**CIFAR100**
- VGG19
| | AUROC(%) | Detection Acc(%) | #(train, dev, test) |
|:-----------|-----------:|-------------------:|:----------------------|
| DeepFool | 80.4 | 73.61 | (1893, 315, 949) |
| BIM | 86.14 | 78.24 | (1879, 312, 943) |
| CW | 72.53 | 68.66 | (1461, 243, 734) |
| Low_CW | 59.76 | 59.38 | (779, 129, 392) |
| FAB | 76.26 | 70.87 | (1836, 305, 920) |
| FGSM | 63.77 | 62.82 | (1247, 206, 627) |
| PGD | 87.85 | 81.94 | (1872, 310, 940) |
| PGD_L2 | 81.07 | 77.37 | (1848, 306, 929) |
| Low_PGD1 | 55.68 | 58.55 | (782, 129, 395) |
| Low_PGD2 | 66.83 | 67.21 | (1483, 246, 745) |
| AutoAttack | 80.02 | 73.1 | (1969, 327, 988) |
| Square | 77.73 | 72.67 | (1859, 309, 932) |
- ResNet34
| | AUROC(%) | Detection Acc(%) | #(train, dev, test) |
|:-----------|-----------:|-------------------:|:----------------------|
| DeepFool | 74.94 | 73.37 | (2118, 351, 1064) |
| BIM | 98.02 | 94.69 | (2749, 457, 1378) |
| CW | 78.52 | 74.3 | (1936, 321, 970) |
| Low_CW | 57.16 | 59.92 | (1019, 169, 514) |
| FAB | 76.46 | 71.85 | (2320, 385, 1165) |
| FGSM | 93.41 | 87.5 | (1817, 302, 912) |
| PGD | 97.23 | 94 | (2652, 441, 1328) |
| PGD_L2 | 97.29 | 94.04 | (2669, 444, 1338) |
| Low_PGD1 | 59.75 | 61.08 | (794, 132, 399) |
| Low_PGD2 | 68.78 | 67.1 | (1925, 320, 964) |
| AutoAttack | 93.93 | 89.83 | (2547, 423, 1277) |
| Square | 85.4 | 79.39 | (2406, 400, 1205) |
**SVHN**
- VGG19
| | AUROC(%) | Detection Acc(%) | #(train, dev, test) |
|:-----------|-----------:|-------------------:|:----------------------|
| DeepFool | 95.4 | 89.52 | (18605, 3100, 9304) |
| BIM | 98.96 | 95.2 | (24104, 4017, 12055) |
| CW | 93.56 | 87.75 | (14674, 2445, 7339) |
| Low_CW | 90.12 | 84.41 | (3358, 559, 1683) |
| FAB | 96.75 | 92.39 | (15473, 2578, 7739) |
| FGSM | 92.78 | 86.19 | (15777, 2628, 7891) |
| PGD | 98.9 | 94.87 | (23855, 3975, 11931) |
| PGD_L2 | 98.67 | 94.37 | (25080, 4179, 12543) |
| Low_PGD1 | 78.78 | 72.43 | (2203, 365, 1106) |
| Low_PGD2 | 88.59 | 81.52 | (7886, 1313, 3948) |
| AutoAttack | 98.94 | 94.77 | (23558, 3925, 11782) |
| Square | 98.3 | 94.83 | (15747, 2624, 7876) |
- ResNet34
| | AUROC(%) | Detection Acc(%) | #(train, dev, test) |
|:-----------|-----------:|-------------------:|:----------------------|
| DeepFool | 94.33 | 88.66 | (16354, 2725, 8180) |
| BIM | 99.57 | 97.43 | (23573, 3927, 11790) |
| CW | 90.29 | 84.11 | (11686, 1947, 5845) |
| Low_CW | 87.5 | 83.36 | (3035, 504, 1523) |
| FAB | 96.35 | 91.42 | (15002, 2499, 7504) |
| FGSM | 89.32 | 82.5 | (12162, 2026, 6084) |
| PGD | 99.49 | 97.21 | (23148, 3857, 11576) |
| PGD_L2 | 98.75 | 94.97 | (24687, 4114, 12347) |
| Low_PGD1 | 81.22 | 76.09 | (2268, 377, 1136) |
| Low_PGD2 | 89.78 | 82.66 | (8342, 1389, 4174) |
| AutoAttack | 99.54 | 97.3 | (22959, 3825, 11482) |
| Square | 98.23 | 95.14 | (15704, 2616, 7855) |
## 3. Transfer Attacks
- **Row**: Source
- **Column**: Target
**CIFAR10**
- VGG19
| | DeepFool | BIM | CW | Low_CW | FAB | FGSM | PGD | PGD_L2 | Low_PGD1 | Low_PGD2 | AutoAttack | Square |
|:-----------|-----------:|------:|------:|---------:|------:|-------:|------:|---------:|-----------:|-----------:|-------------:|---------:|
| DeepFool | 91.76 | 61.91 | 90.83 | 90.47 | 89.7 | 89.07 | 62.33 | 60.69 | 84.43 | 84.71 | 62.17 | 89.83 |
| BIM | 65.76 | 99.79 | 81.7 | 41.25 | 43.87 | 82.32 | 99.78 | 99.81 | 63.43 | 96.45 | 99.79 | 32.8 |
| CW | 88.16 | 69.72 | 88.31 | 87.83 | 84.78 | 88.94 | 71.69 | 69.97 | 80.25 | 86.33 | 71.58 | 85.77 |
| Low_CW | 89.16 | 51.94 | 85.72 | 90.99 | 87.82 | 80.62 | 51.36 | 50.84 | 88.69 | 75.91 | 55.31 | 90.16 |
| FAB | 95.84 | 43.01 | 93.17 | 92.47 | 97.3 | 91.51 | 44.71 | 43.09 | 82.6 | 76.42 | 45.86 | 96.72 |
| FGSM | 84.7 | 72.84 | 88.67 | 81.27 | 79.18 | 90.67 | 75.32 | 73.41 | 74.65 | 84.2 | 73.15 | 83.41 |
| PGD | 66.58 | 99.82 | 81.71 | 42.17 | 45.08 | 84.06 | 99.67 | 99.78 | 62.75 | 96.62 | 99.77 | 36.41 |
| PGD_L2 | 67.39 | 99.83 | 84.48 | 46.9 | 48.83 | 85.01 | 99.77 | 99.74 | 69.58 | 96.98 | 99.81 | 36.83 |
| Low_PGD1 | 77.68 | 69.07 | 80.04 | 82.66 | 77.93 | 77.01 | 69.56 | 68.64 | 80.74 | 81.79 | 71.75 | 76.51 |
| Low_PGD2 | 77.36 | 90.23 | 83.99 | 71.57 | 68.34 | 83.23 | 90.06 | 90.25 | 76.74 | 89.74 | 90.26 | 66.8 |
| AutoAttack | 66.91 | 99.72 | 82.04 | 38.63 | 43.62 | 85.85 | 99.66 | 99.7 | 63.95 | 95.72 | 99.64 | 38.06 |
| Square | 95.91 | 47.4 | 94.39 | 96.43 | 96.83 | 92.97 | 48.66 | 46.95 | 87.56 | 84.2 | 48.42 | 97.59 |
- ResNet34
| | DeepFool | BIM | CW | Low_CW | FAB | FGSM | PGD | PGD_L2 | Low_PGD1 | Low_PGD2 | AutoAttack | Square |
|:-----------|-----------:|------:|------:|---------:|------:|-------:|------:|---------:|-----------:|-----------:|-------------:|---------:|
| DeepFool | 92.94 | 73.1 | 89.64 | 86.91 | 94.01 | 84.39 | 75.35 | 71.03 | 87.59 | 91.4 | 74.82 | 90.98 |
| BIM | 62.13 | 99.39 | 83.62 | 56.03 | 60.76 | 76.35 | 99.32 | 99.45 | 66 | 90.74 | 99.33 | 45.59 |
| CW | 87.24 | 81.21 | 90.28 | 81.74 | 84.14 | 88.61 | 80.69 | 79.03 | 83.92 | 87.26 | 82.65 | 86.05 |
| Low_CW | 84.41 | 62.76 | 84.19 | 82.58 | 85.55 | 72.83 | 67.99 | 58.3 | 88.52 | 84.09 | 63.57 | 85.86 |
| FAB | 88.69 | 74.51 | 86.99 | 81.15 | 95.76 | 84.35 | 79.18 | 76.58 | 78.78 | 90.76 | 80.11 | 86.79 |
| FGSM | 86.49 | 82.09 | 92.81 | 83.56 | 82.92 | 94.89 | 81.67 | 81.75 | 81.56 | 86.81 | 83.67 | 85.67 |
| PGD | 59.23 | 99.42 | 82.57 | 57.65 | 56.27 | 75.99 | 99.28 | 99.36 | 64.2 | 90.35 | 99.46 | 44.45 |
| PGD_L2 | 65.7 | 99.1 | 83.43 | 61.45 | 64.05 | 73.18 | 99.04 | 98.88 | 67.84 | 91.59 | 98.81 | 54.11 |
| Low_PGD1 | 77.44 | 70.08 | 82.17 | 79.2 | 79.07 | 75.33 | 77.02 | 69.89 | 82.95 | 86.42 | 74.85 | 81.79 |
| Low_PGD2 | 85.74 | 92.84 | 90.4 | 80.67 | 86.79 | 85.36 | 91.48 | 92.35 | 85.76 | 90.07 | 92.79 | 82.17 |
| AutoAttack | 66.04 | 98 | 82.31 | 61.45 | 58.13 | 79.81 | 97.69 | 98.18 | 64.44 | 89.35 | 97.57 | 51.35 |
| Square | 92.02 | 63.15 | 93.01 | 86.4 | 94.09 | 85.05 | 68.54 | 62.23 | 87.73 | 91.03 | 68.51 | 95.32 |
**CIFAR100**
- VGG19
| | DeepFool | BIM | CW | Low_CW | FAB | FGSM | PGD | PGD_L2 | Low_PGD1 | Low_PGD2 | AutoAttack | Square |
|:-----------|-----------:|------:|------:|---------:|------:|-------:|------:|---------:|-----------:|-----------:|-------------:|---------:|
| DeepFool | 80.4 | 52.95 | 71.77 | 81.31 | 82.27 | 68.61 | 51.34 | 47.03 | 72.65 | 70.04 | 51.3 | 77.77 |
| BIM | 46.63 | 86.14 | 57.88 | 53.55 | 41.39 | 61.81 | 90.2 | 84.8 | 56.72 | 73.45 | 80.44 | 42.52 |
| CW | 75.26 | 72.3 | 72.53 | 69.18 | 72.6 | 67.45 | 65.02 | 64.75 | 66.54 | 71.74 | 64.16 | 68.25 |
| Low_CW | 83.81 | 52.64 | 83.21 | 59.76 | 78.18 | 60.5 | 52.16 | 54.37 | 62.42 | 72.59 | 52.83 | 65.95 |
| FAB | 84.15 | 53.52 | 72.35 | 74.6 | 76.26 | 58.67 | 54.37 | 50.09 | 65.21 | 66.54 | 53.59 | 76.95 |
| FGSM | 72.34 | 61.55 | 71.93 | 67.63 | 71.73 | 63.77 | 62.16 | 59.52 | 63.21 | 64.98 | 58.82 | 71.91 |
| PGD | 44.98 | 89.12 | 51.08 | 45.1 | 40.51 | 56.23 | 87.85 | 85.88 | 52.43 | 66.13 | 81.35 | 44.49 |
| PGD_L2 | 41.47 | 89.16 | 55.15 | 43.89 | 38.49 | 49.78 | 91.12 | 81.07 | 59.44 | 70.08 | 79.57 | 41.95 |
| Low_PGD1 | 76.1 | 57.68 | 73.99 | 75.12 | 77.17 | 61.2 | 62.45 | 68.64 | 55.68 | 75.67 | 55.72 | 67.65 |
| Low_PGD2 | 64.91 | 77.37 | 64.82 | 62.86 | 59.45 | 60.75 | 73.54 | 70.8 | 70.4 | 66.83 | 66.56 | 60.75 |
| AutoAttack | 49.88 | 89.22 | 58.27 | 45.34 | 42.91 | 61.9 | 88.54 | 83.86 | 54.29 | 65.01 | 80.02 | 45.25 |
| Square | 81.28 | 51.62 | 78.49 | 75.29 | 83.78 | 65.03 | 55.67 | 51.96 | 68.4 | 70.97 | 55.75 | 77.73 |
- ResNet34
| | DeepFool | BIM | CW | Low_CW | FAB | FGSM | PGD | PGD_L2 | Low_PGD1 | Low_PGD2 | AutoAttack | Square |
|:-----------|-----------:|------:|------:|---------:|------:|-------:|------:|---------:|-----------:|-----------:|-------------:|---------:|
| DeepFool | 74.94 | 57.48 | 75.73 | 73.52 | 83.33 | 72.89 | 54.86 | 58.38 | 67.29 | 67.29 | 59.76 | 79.52 |
| BIM | 31.34 | 98.02 | 55.44 | 38.47 | 28.08 | 50.25 | 95.76 | 97.58 | 31.97 | 72.02 | 93.13 | 39.29 |
| CW | 77.76 | 72.18 | 78.52 | 76.02 | 74.37 | 79.36 | 68.92 | 71.47 | 69.53 | 73.97 | 68.14 | 73.56 |
| Low_CW | 78.28 | 72.04 | 84.22 | 57.16 | 78.3 | 64.95 | 60.15 | 67.52 | 71.12 | 72.49 | 64.14 | 73.62 |
| FAB | 78.39 | 58.57 | 76.55 | 66.73 | 76.46 | 74.02 | 58.53 | 62.44 | 65.64 | 71.09 | 61.52 | 78.15 |
| FGSM | 77.61 | 71.77 | 88.83 | 70.72 | 79.05 | 93.41 | 68.76 | 70.19 | 65.67 | 69.9 | 77.14 | 84.17 |
| PGD | 26.7 | 98.19 | 57.63 | 36.64 | 31.42 | 52.43 | 97.23 | 99.56 | 34.39 | 75.72 | 95.32 | 37.67 |
| PGD_L2 | 28.46 | 98.88 | 51.37 | 35.31 | 25.86 | 46.27 | 95.76 | 97.29 | 29.97 | 73.22 | 92.96 | 32.25 |
| Low_PGD1 | 72.63 | 72.77 | 80.75 | 72.64 | 64.63 | 65.4 | 66.8 | 73.77 | 59.75 | 81.46 | 73.27 | 70.26 |
| Low_PGD2 | 64.37 | 83.87 | 79.34 | 68.54 | 67.23 | 68.19 | 80.81 | 82.5 | 60 | 68.78 | 85.91 | 68.7 |
| AutoAttack | 28.99 | 98 | 59.05 | 37.38 | 32.47 | 61.76 | 98.28 | 99.2 | 31.78 | 72.65 | 93.93 | 47.21 |
| Square | 81.09 | 66.76 | 75.13 | 70.5 | 80.85 | 74.05 | 66.64 | 72.78 | 55.91 | 74.63 | 72.1 | 85.4 |
**SVHN**
- VGG19
| | DeepFool | BIM | CW | Low_CW | FAB | FGSM | PGD | PGD_L2 | Low_PGD1 | Low_PGD2 | AutoAttack | Square |
|:-----------|-----------:|------:|------:|---------:|------:|-------:|------:|---------:|-----------:|-----------:|-------------:|---------:|
| DeepFool | 95.4 | 35.78 | 95.32 | 94.09 | 94.48 | 95.07 | 33.91 | 33.6 | 90.72 | 82.15 | 39.71 | 93.9 |
| BIM | 32.3 | 98.96 | 31.01 | 22.1 | 34.79 | 31.1 | 99.05 | 98.99 | 55.46 | 91.36 | 99.06 | 16.17 |
| CW | 93.02 | 40.19 | 93.56 | 92.91 | 92.86 | 93.21 | 38.05 | 37.82 | 90.77 | 85.13 | 44.09 | 92.19 |
| Low_CW | 86.67 | 47.91 | 87.49 | 90.12 | 86.96 | 84.54 | 44.6 | 43.49 | 87.35 | 78.68 | 50.73 | 89.86 |
| FAB | 95.67 | 40.36 | 95.64 | 95.64 | 96.75 | 95.33 | 38.83 | 37.38 | 91.82 | 83.18 | 44.95 | 96.3 |
| FGSM | 92.08 | 38.91 | 92.23 | 90.46 | 91.06 | 92.78 | 36.37 | 35.97 | 87.41 | 80.92 | 42.57 | 89.7 |
| PGD | 33.59 | 98.85 | 32.33 | 23.09 | 35.95 | 32.42 | 98.9 | 98.92 | 55.99 | 91.46 | 98.96 | 17.7 |
| PGD_L2 | 31.62 | 98.65 | 28.87 | 20.87 | 32.87 | 29.1 | 98.69 | 98.67 | 54.7 | 91.03 | 98.66 | 16.58 |
| Low_PGD1 | 72.79 | 63.26 | 74.54 | 76.43 | 70.4 | 69.07 | 59.46 | 62.38 | 78.78 | 77.47 | 64.46 | 71.02 |
| Low_PGD2 | 76.74 | 77.22 | 79.7 | 75.42 | 76.82 | 77.04 | 77.11 | 77.54 | 83.25 | 88.59 | 78.72 | 71.01 |
| AutoAttack | 38.41 | 98.85 | 37.05 | 25.45 | 40.35 | 37.31 | 98.96 | 98.89 | 58.59 | 91.69 | 98.94 | 19.92 |
| Square | 96.99 | 33.77 | 96.93 | 97.95 | 97.11 | 95.84 | 29.17 | 30.33 | 94.99 | 87.48 | 34.8 | 98.3 |
- ResNet34
| | DeepFool | BIM | CW | Low_CW | FAB | FGSM | PGD | PGD_L2 | Low_PGD1 | Low_PGD2 | AutoAttack | Square |
|:-----------|-----------:|------:|------:|---------:|------:|-------:|------:|---------:|-----------:|-----------:|-------------:|---------:|
| DeepFool | 94.33 | 53.61 | 93.78 | 89.5 | 91.69 | 91.08 | 55.08 | 57.89 | 86.06 | 77.35 | 52.81 | 92.63 |
| BIM | 34.16 | 99.57 | 46.25 | 23.13 | 24.32 | 55.06 | 99.63 | 99.59 | 55.35 | 94.76 | 99.56 | 15.03 |
| CW | 88.69 | 58.87 | 90.29 | 86.35 | 86.55 | 88.29 | 61.13 | 62.77 | 84.75 | 79.73 | 59.11 | 86.49 |
| Low_CW | 87.76 | 54.83 | 87.18 | 87.5 | 87.62 | 80.5 | 57.36 | 54.97 | 87.41 | 80 | 57.78 | 89.15 |
| FAB | 96.59 | 56.24 | 95.28 | 93.84 | 96.35 | 92.54 | 58.97 | 57.76 | 88.04 | 78.36 | 56.18 | 97.06 |
| FGSM | 85.22 | 60.89 | 87.28 | 81.43 | 83.11 | 89.32 | 62.84 | 63.46 | 78.65 | 74.61 | 59.8 | 83.29 |
| PGD | 35.69 | 99.55 | 48.53 | 24.24 | 25.97 | 55.67 | 99.49 | 99.51 | 55.93 | 94.37 | 99.4 | 16.04 |
| PGD_L2 | 34.54 | 98.55 | 44.23 | 24.16 | 25.42 | 52.18 | 98.56 | 98.75 | 53.97 | 92.27 | 98.27 | 16.5 |
| Low_PGD1 | 79.42 | 66.12 | 81.67 | 80.15 | 76.53 | 75.8 | 64.34 | 66.59 | 81.22 | 81.41 | 68.53 | 75.92 |
| Low_PGD2 | 73.31 | 84.54 | 80.64 | 70.08 | 68.54 | 76.95 | 84.6 | 85.39 | 81.43 | 89.78 | 84.51 | 64.15 |
| AutoAttack | 38.17 | 99.61 | 50.6 | 23.6 | 26.74 | 57.42 | 99.55 | 99.55 | 55.86 | 94.62 | 99.54 | 16.53 |
| Square | 97.49 | 56.05 | 96.95 | 96.46 | 96.03 | 94.34 | 59.29 | 57.35 | 93.28 | 83.35 | 57.63 | 98.23 |
# Citations
```bibtex
@article{kim2020torchattacks,
title={Torchattacks: A pytorch repository for adversarial attacks},
author={Kim, Hoki},
journal={arXiv preprint arXiv:2010.01950},
year={2020}
}
```