https://github.com/fra31/robust-finetuning
Code relative to "Adversarial robustness against multiple and single $l_p$-threat models via quick fine-tuning of robust classifiers"
https://github.com/fra31/robust-finetuning
adversarial-learning adversarial-robustness adversarial-training
Last synced: 14 days ago
JSON representation
Code relative to "Adversarial robustness against multiple and single $l_p$-threat models via quick fine-tuning of robust classifiers"
- Host: GitHub
- URL: https://github.com/fra31/robust-finetuning
- Owner: fra31
- Created: 2021-05-25T11:28:20.000Z (almost 4 years ago)
- Default Branch: master
- Last Pushed: 2022-11-30T13:39:50.000Z (over 2 years ago)
- Last Synced: 2024-08-04T03:11:05.846Z (10 months ago)
- Topics: adversarial-learning, adversarial-robustness, adversarial-training
- Language: Python
- Homepage: https://arxiv.org/abs/2105.12508
- Size: 19.5 KB
- Stars: 15
- Watchers: 2
- Forks: 4
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Robust fine-tuning
"Adversarial robustness against multiple and single $l_p$-threat models via quick fine-tuning of robust classifiers"\
*Francesco Croce, Matthias Hein*\
ICML 2022\
[https://arxiv.org/abs/2105.12508](https://arxiv.org/abs/2105.12508)
We propose to *i)* use adversarial training wrt Linf and L1 (alternating the two threat models) to achieve robustness also to L2 and *ii)* fine-tune models robust in one
Lp-norm to get multiple norm robustness or robustness wrt another Lq-norm.
## Code
### Training code
The file `train.py` allows to train or fine-tune models. For adversarial training use `--attack=apgd`, otherwise standard training is performed. The main arguments
for adversarial training are (other options in `train.py`)
+ `--l_norms='Linf L1'`, the list (as string with blank space separated items) of Lp-norms, even just one, to use for training (note that the training cost is the same
regardless of the number of threat models used),
+ `--l_eps`, list of thresholds epsilon for each threat model for training (if not given, the default values are used), sorted as the corresponding norms.
+ `--l_iters`, list of iterations in adversarial training for each threat model (possibly different), or `--at_iter`, number of steps for all threat models.
For training new models a PreAct ResNet-18 is used, by default with softplus activation function.
### Fine-tuning existing models
To fine-tune a model add the `--finetune_model` flag, `--lr-schedule=piecewise-ft` to set the standard learning rate schedule,
`--model_dir=/path/to/pretrained/models` where to download or find the models.
+ We provide [here](https://drive.google.com/drive/folders/1hYWHp5UbTAm9RhSb8JkJZtcB0LDZDvkT?usp=sharing) pre-trained ResNet-18 robust wrt Linf, L2 and L1,
which can be loaded specifying `--model_name=pretr_L*.pth` (insert the desired norm).
+ It is also possible to use models from the [Model Zoo](https://github.com/RobustBench/robustbench#model-zoo) of [RobustBench](https://robustbench.github.io/)
with `--model_name=RB_{}` inserting the identifier of the classifier from the Model Zoo (these are automatically downloaded). Note that models trained with extra data should be fine-tuned with the same
(currently not supported in the code).
### Evaluation code
With `--final_eval` our standard evaluation (with APGD-CE and APGD-T, for a total of 10 restarts of 100 steps) is run for all threat models at the end of training.
Specifying `--eval_freq=k` a fast evaluation is run on test and training points every `k` epochs.
To evaluate a trained model one can run `eval.py` with `--model_name` as above for the pretrained model or `--model_name=/path/to/checkpoint/` for new or fine-tuned
classifiers. If the run has the automatically generated name, the corresponding architecture is loaded. More details about the options for evaluation in `eval.py`.
## Credits
Parts of the code in this repo is based on
+ [https://github.com/tml-epfl/understanding-fast-adv-training](https://github.com/tml-epfl/understanding-fast-adv-training)
+ [https://github.com/locuslab/robust_overfitting](https://github.com/locuslab/robust_overfitting)