{"id":13487404,"url":"https://github.com/P2333/Papers-of-Robust-ML","last_synced_at":"2025-03-27T22:31:02.682Z","repository":{"id":36488818,"uuid":"161721422","full_name":"P2333/Papers-of-Robust-ML","owner":"P2333","description":" Related papers for robust machine learning","archived":false,"fork":false,"pushed_at":"2023-05-25T10:34:48.000Z","size":207,"stargazers_count":563,"open_issues_count":0,"forks_count":55,"subscribers_count":43,"default_branch":"master","last_synced_at":"2024-08-01T18:30:10.878Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/P2333.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-12-14T02:31:53.000Z","updated_at":"2024-06-13T08:16:51.000Z","dependencies_parsed_at":"2024-01-16T09:00:58.479Z","dependency_job_id":"d97c6b57-ae85-4814-a4db-eef0f48f7f45","html_url":"https://github.com/P2333/Papers-of-Robust-ML","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/P2333%2FPapers-of-Robust-ML","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/P2333%2FPapers-of-Robust-ML/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/P2333%2FPapers-of-Robust-ML/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/P2333%2FPapers-of-Robust-ML/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/P2333","download_url":"https://codeload.github.com/P2333/Papers-of-Robust-ML/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222322028,"owners_count":16966433,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T18:00:58.884Z","updated_at":"2024-10-30T22:31:03.340Z","avatar_url":"https://github.com/P2333.png","language":null,"funding_links":[],"categories":["Robustness Lists","Please find below the links to awesome cheat-sheet and resources:","Table of Contents"],"sub_categories":["Machine-Learning/Data Science/AI/DL:"],"readme":"# Papers-of-Robust-ML\nRelated papers for robust machine learning (we mainly focus on defenses).\n \n# Statement\nSince there are tens of new papers on adversarial defense in each conference, we are only able to update those we just read and consider as insightful.\n\nAnyone is welcomed to submit a pull request for the related and unlisted papers on adversarial defense, which are pulished on peer-review conferences (ICML/NeurIPS/ICLR/CVPR etc.) or released on arXiv.\n\n## Contents \n- \u003ca href=\"#General_training\"\u003eGeneral Defenses (training phase)\u003c/a\u003e\u003cbr\u003e\n- \u003ca href=\"#General_inference\"\u003eGeneral Defenses (inference phase)\u003c/a\u003e\u003cbr\u003e\n- \u003ca href=\"#Detection\"\u003eAdversarial Detection\u003c/a\u003e\u003cbr\u003e\n- \u003ca href=\"#Certified Defense and Model Verification\"\u003eCertified Defense and Model Verification\u003c/a\u003e\u003cbr\u003e\n- \u003ca href=\"#Theoretical\"\u003eTheoretical Analysis\u003c/a\u003e\u003cbr\u003e\n- \u003ca href=\"#Empirical\"\u003eEmpirical Analysis\u003c/a\u003e\u003cbr\u003e\n- \u003ca href=\"#Beyond_Safety\"\u003eBeyond Safety (Adversarial for Good)\u003c/a\u003e\u003cbr\u003e\n- \u003ca href=\"#Seminal_work\"\u003eSeminal Work\u003c/a\u003e\u003cbr\u003e\n- \u003ca href=\"#Benchmark_Datasets\"\u003eBenchmark Datasets\u003c/a\u003e\u003cbr\u003e\n\n\n\u003ca id='General_training'\u003e\u003c/a\u003e\n## General Defenses (training phase)\n* [Better Diffusion Models Further Improve Adversarial Training](https://arxiv.org/pdf/2302.04638.pdf) (ICML 2023) \u003cbr/\u003e This paper advocate that better diffusion models such as EDM can further improve adversarial training beyond using DDPM, which achieves new state-of-the-art performance on CIFAR-10/100 as listed on RobustBench.\n\n* [FrequencyLowCut Pooling -- Plug \u0026 Play against Catastrophic Overfitting](https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136740036.pdf) (ECCV 2022) \u003cbr/\u003e This paper proposes a novel aliasing-free downsampling layer to prevent catastrophic overfitting during simple Fast Gradient Sign Method (FGSM) adversarial training. \n\n* [Robustness and Accuracy Could Be Reconcilable by (Proper) Definition](https://arxiv.org/pdf/2202.10103.pdf) (ICML 2022) \u003cbr/\u003e This paper advocate that robustness and accuracy are not at odds, as long as we slightly modify the definition of robust error. Efficient ways of optimizating the new SCORE objective is provided.\n\n* [Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attacks](https://openreview.net/pdf?id=9CPc4EIr2t1) (NeurIPS 2021) \u003cbr/\u003e This paper combines the stable conditions in control theory into neural ODE to induce locally stable models. \n\n* [Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart ](https://arxiv.org/pdf/2105.14785.pdf) (CVPR 2022) \u003cbr/\u003e This paper proposes a coupling rejection strategy, where two simple but well-designed rejection metrics can be coupled to provabably distinguish any misclassified sample from correclty classified ones.\n\n* [Fixing Data Augmentation to Improve Adversarial Robustness](https://arxiv.org/pdf/2103.01946.pdf) (NeurIPS 2021) \u003cbr/\u003e This paper shows that after applying weight moving average, data augmentation (either by transformatons or generative models) can further improve robustness of adversarial training.\n\n* [Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?](https://arxiv.org/pdf/2104.09425.pdf) (ICLR 2022) \u003cbr/\u003e This paper verifies that leveraging more data sampled from a (high-quality) generative model that was trained on the same dataset (e.g., CIFAR-10) can still improve robustness of adversarially trained models, without using any extra data.\n\n* [Towards Robust Neural Networks via Close-loop Control](https://openreview.net/forum?id=2AL06y9cDE-) (ICLR 2021) \u003cbr/\u003e This paper introduce a close-loop control framework to enhance adversarial robustness of trained networks.\n\n* [Understanding and Improving Fast Adversarial Training](https://arxiv.org/pdf/2007.02617.pdf) (NeurIPS 2020) \u003cbr/\u003e A systematic study of catastrophic overfitting in adversarial training, its reasons, and ways of resolving it. The proposed regularizer, *GradAlign*, helps to prevent catastrophic overfitting and scale FGSM training to high Linf-perturbations.\n\n* [Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks](https://arxiv.org/pdf/1910.06259.pdf) (ICML 2020) \u003cbr/\u003e This paper uses a perturbation-dependent label smoothing method to generalize adversarially trained models to unseen attacks.\n\n* [Smooth Adversarial Training](https://arxiv.org/pdf/2006.14536.pdf) \u003cbr/\u003e This paper advocate using smooth variants of ReLU during adversarial training, which can achieve state-of-the-art performance on ImageNet.  \n\n* [Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness](https://openreview.net/forum?id=Byg9A24tvB) (ICLR 2020) \u003cbr/\u003e This paper rethink the drawbacks of softmax cross-entropy in the adversarial setting, and propose the MMC method to induce high-density regions in the feature space.\n\n* [Jacobian Adversarially Regularized Networks for Robustness](https://openreview.net/pdf?id=Hke0V1rKPS) (ICLR 2020) \u003cbr/\u003e This paper propose to show that a generally more interpretable model could potentially be more robust against adversarial attacks.\n\n* [Fast is better than free: Revisiting adversarial training](https://openreview.net/forum?id=BJx040EFvH\u0026noteId=BJx040EFvH) (ICLR 2020) \u003cbr/\u003e This paper proposes several tricks to make FGSM-based adversarial training effective.\n\n* [Adversarial Training and Provable Defenses: Bridging the Gap](https://openreview.net/forum?id=SJxSDxrKDr) (ICLR 2020) \u003cbr/\u003e This paper proposes the layerwise adversarial training method, which gradually optimizes on the latent adversarial examples from low-level to high-level layers.\n\n* [Improving Adversarial Robustness Requires Revisiting Misclassified Examples](https://openreview.net/forum?id=rklOg6EFwS) (ICLR 2020) \u003cbr/\u003e This paper proposes a new method MART, which involves a boosted CE loss to further lower down the second-maximal prediction, and a weighted KL term (similar as a focal loss), compared to the formula of TRADES.\n\n* [Adversarial Interpolation Training: A Simple Approach for Improving Model Robustness](https://openreview.net/forum?id=Syejj0NYvr\u0026noteId=r1e432RzoS) \u003cbr/\u003e This paper introduces the mixup method into adversarial training to improve the model performance on clean images.\n\n* [Are labels required for improving adversarial robustness?](https://arxiv.org/pdf/1905.13725.pdf) (NeurIPS 2019) \u003cbr/\u003e This paper exploit unlabeled data to better improve adversarial robustness.\n\n* [Adversarial Robustness through Local Linearization](https://arxiv.org/pdf/1907.02610.pdf) (NeurIPS 2019) \u003cbr/\u003e This paper introduce local linearization in adversarial training process.\n\n* [Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks](https://arxiv.org/pdf/1906.03526.pdf) (NeurIPS 2019) \u003cbr/\u003e A method to efficiently certify the robustness of GBDTs and to integrate the certificate into training (leads to an upper bound on the worst-case loss). The obtained certified accuracy is higher than for other robust GBDTs and is competitive to provably robust CNNs.\n\n* [You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle](https://arxiv.org/pdf/1905.00877.pdf) (NeurIPS 2019) \u003cbr/\u003e This paper provides a fast method for adversarial training from the perspective of optimal control.\n\n* [Adversarial Training for Free!](https://arxiv.org/pdf/1904.12843.pdf) (NeurIPS 2019) \u003cbr/\u003e A fast method for adversarial training, which shares the back-propogation gradients of updating weighs and crafting adversarial examples.\n\n* [ME-Net: Towards Effective Adversarial Robustness with Matrix Estimation](https://arxiv.org/abs/1905.11971) (ICML 2019) \u003cbr/\u003e This paper demonstrates the global low-rank structures within images, and leverages matrix estimation to exploit such underlying structures for better adversarial robustness.\n\n* [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://arxiv.org/abs/1901.09960) (ICML 2019) \u003cbr/\u003e\nThis paper shows adversarial robustness can transfer and that adversarial pretraining can increase adversarial robustness by ~10% accuracy.\n\n* [Theoretically Principled Trade-off between Robustness and Accuracy](https://arxiv.org/pdf/1901.08573.pdf) (ICML 2019) \u003cbr/\u003e A variant of adversarial training: TRADES, which won the defense track of NeurIPS 2018 Adversarial Competation.\n\n* [Robust Decision Trees Against Adversarial Examples](http://web.cs.ucla.edu/~chohsieh/ICML_2019_TreeAdvAttack.pdf) (ICML 2019) \u003cbr/\u003e A method to enhance the robustness of tree models, including GBDTs.\n\n* [Improving Adversarial Robustness via Promoting Ensemble Diversity](https://arxiv.org/pdf/1901.08846.pdf) (ICML 2019) \u003cbr/\u003e Previous work constructs ensemble defenses by individually enhancing each memeber and then directly average the predictions. In this work, the authors propose the adaptive diversity promoting (ADP) to further improve the robustness by promoting the ensemble diveristy, as an orthogonal methods compared to other defenses.\n\n* [Feature Denoising for Improving Adversarial Robustness](https://arxiv.org/pdf/1812.03411.pdf) (CVPR 2019) \u003cbr/\u003e This paper applies non-local neural network and large-scale adversarial training with 128 GPUs (with training trick in 'Accurate, large minibatch SGD: Training ImageNet in 1 hour'), which shows large improvement than previous SOTA trained with 50 GPUs.\n\n* [Improving the Generalization of Adversarial Training with Domain Adaptation](https://arxiv.org/pdf/1810.00740.pdf) (ICLR 2019) \u003cbr/\u003e This work proposes to use additional regularization terms to match the domains between clean and adversarial logits in adversarial training.\n\n* [A Spectral View of Adversarially Robust Features](http://papers.nips.cc/paper/8217-a-spectral-view-of-adversarially-robust-features.pdf) (NeurIPS 2018) \u003cbr/\u003e Given the entire dataset X, use the eigenvectors of spectral graph as robust features. [[Appendix](http://papers.nips.cc/paper/8217-a-spectral-view-of-adversarially-robust-features-supplemental.zip)]\n\n* [Adversarial Logit Pairing](https://arxiv.org/pdf/1803.06373.pdf) \u003cbr/\u003e Adversarial training by pairing the clean and adversarial logits.\n\n* [Deep Defense: Training DNNs with Improved Adversarial Robustness](http://papers.nips.cc/paper/7324-deep-defense-training-dnns-with-improved-adversarial-robustness.pdf) (NeurIPS 2018) \u003cbr/\u003e They follow the linear assumption in DeepFool method. DeepDefense pushes decision boundary away from those correctly classified, and pull decision boundary closer to those misclassified.\n\n* [Max-Mahalanobis Linear Discriminant Analysis Networks](http://proceedings.mlr.press/v80/pang18a/pang18a.pdf) (ICML 2018) \u003cbr/\u003e This is one of our work. We explicitly model the feature distribution as a Max-Mahalanobis distribution (MMD), which has max margin among classes and can lead to guaranteed robustness.\n\n* [Ensemble Adversarial Training- Attacks and Defenses](https://arxiv.org/pdf/1705.07204.pdf) (ICLR 2018) \u003cbr/\u003e Ensemble adversarial training use sevel pre-trained models, and in each training batch, they randomly select one of the currently trained model or pre-trained models to craft adversarial examples.\n\n* [Pixeldefend: Leveraging generative models to understand and defend against adversarial examples](https://arxiv.org/abs/1710.10766) (ICLR 2018) \u003cbr/\u003e This paper provided defense by moving adversarial examples back towards the distribution seen in the training data.\n\n\u003ca id='General_inference'\u003e\u003c/a\u003e\n## General Defenses (inference phase)\n* [Adversarial Attacks are Reversible with Natural Supervision](https://arxiv.org/abs/2103.14222) (ICCV 2021) \u003cbr/\u003e This paper proposes to use contrastive loss to restore the natural structure of attacked images, providing a defense.\n\n* [Adversarial Purification with Score-based Generative Models](https://arxiv.org/pdf/2106.06041.pdf) (ICML 2021) \u003cbr/\u003e This paper proposes to use score-based generative models (e.g., NCSN) to purify adversarial examples.\n\n* [Online Adversarial Purification based on Self-Supervision](https://arxiv.org/abs/2101.09387) (ICLR 2021) \u003cbr/\u003e This paper proposes to train the network with a label-independent auxiliary task (e.g., rotation prediction), and purify the test inputs dynamically by minimizing the auxiliary loss.\n\n* [Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks](https://openreview.net/forum?id=ByxtC2VtPB) (ICLR 2020) \u003cbr/\u003e This paper exploit the mixup mechanism in the inference phase to improve robustness.\n\n* [Barrage of Random Transforms for Adversarially Robust Defense](http://openaccess.thecvf.com/content_CVPR_2019/papers/Raff_Barrage_of_Random_Transforms_for_Adversarially_Robust_Defense_CVPR_2019_paper.pdf) (CVPR 2019) \u003cbr/\u003e This paper applies a set of different random transformations as an off-the-shelf defense.\n\n* [Mitigating Adversarial Effects Through Randomization](https://arxiv.org/pdf/1711.01991.pdf) (ICLR 2018) \u003cbr/\u003e Use random resizing and random padding to disturb adversarial examples, which won the 2nd place in th defense track of NeurIPS 2017 Adversarial Competation.\n\n* [Countering Adversarial Images Using Input Transformations](https://arxiv.org/pdf/1711.00117.pdf) (ICLR 2018) \u003cbr/\u003e Apply bit-depth reduction, JPEG compression, total variance minimization and image quilting as input preprocessing to defend adversarial attacks.\n\n\u003ca id='Detection'\u003e\u003c/a\u003e\n## Adversarial Detection\n* [Detecting adversarial examples is (nearly) as hard as classifying them](https://proceedings.mlr.press/v162/tramer22a.html) (ICML 2022) \u003cbr/\u003e This paper demonstrates that detection and classification of adversarial examples can be mutually converted, and thus many previous works on detection may overclaim their effectiveness.\n\n* [Class-Disentanglement and Applications in Adversarial Detection and Defense](https://openreview.net/pdf?id=jFMzBeLyTc0) (NeurIPS 2021) \u003cbr/\u003e This paper proposes to disentangle the class-dependence and visually reconstruction, and exploit the result as an adversarial detection metric.\n\n* [Towards Robust Detection of Adversarial Examples](http://papers.nips.cc/paper/7709-towards-robust-detection-of-adversarial-examples.pdf) (NeurIPS 2018) \u003cbr/\u003e This is one of our work. We train the networks with reverse cross-entropy (RCE), which can map normal features to low-dimensional manifolds, and then detectors can better separate between adversarial examples and normal ones.\n\n* [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](http://papers.nips.cc/paper/7947-a-simple-unified-framework-for-detecting-out-of-distribution-samples-and-adversarial-attacks.pdf) (NeurIPS 2018) \u003cbr/\u003e Fit a GDA on learned features, and use Mahalanobis distance as the detection metric.\n\n* [Robust Detection of Adversarial Attacks by Modeling the Intrinsic Properties of Deep Neural Networks](http://papers.nips.cc/paper/8016-robust-detection-of-adversarial-attacks-by-modeling-the-intrinsic-properties-of-deep-neural-networks.pdf) (NeurIPS 2018) \u003cbr/\u003e They fit a GMM on learned features, and use the probability as the detection metric.\n\n* [Detecting adversarial samples from artifacts](https://arxiv.org/abs/1703.00410) \u003cbr/\u003e This paper proposed the kernel density (K-density) metric on the learned features to detect adversarial examples.\n\n\u003ca id='Certified Defense and Model Verification'\u003e\u003c/a\u003e\n## Certified Defense and Model Verification\n* [Towards Better Understanding of Training Certifiably Robust Models against Adversarial Examples](https://openreview.net/pdf?id=b18Az57ioHn) (NeurIPS 2021) \u003cbr/\u003e  This paper generally study the effciency of different certified defenses, and find that the smoothness of loss landscape matters.\n\n* [Towards Verifying Robustness of Neural Networks against Semantic Perturbations](https://arxiv.org/abs/1912.09533) (CVPR 2020) \u003cbr/\u003e This paper generalize the pixel-wise verification methods into the semantic transformation space.\n\n* [Neural Network Branching for Neural Network Verification](https://arxiv.org/abs/1912.01329) (ICLR 2020) \u003cbr/\u003e This paper use GNN to adaptively construct branching strategy for model verification.\n\n* [Towards Stable and Efficient Training of Verifiably Robust Neural Networks](https://openreview.net/forum?id=Skxuk1rFwB) (ICLR 2020) \u003cbr/\u003e This paper combines the previous IBP and CROWN methods.\n\n* [A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks](http://papers.nips.cc/paper/9176-a-convex-relaxation-barrier-to-tight-robustness-verification-of-neural-networks.pdf) (NeurIPS 2019) \u003cbr/\u003e This paper makes a conprehensive studies on existing robustness verification methods based on convex relaxation.\n\n* [Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers](https://guanghelee.github.io/pub/Lee_etal_neurips19.pdf) (NeurIPS 2019) \u003cbr/\u003e This word extends the robustness certificate of random smoothing from L2 to L0 norm bound.\n\n* [On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models](https://arxiv.org/pdf/1810.12715.pdf) (ICCV 2019) \u003cbr/\u003e This paper proposes the scalable verificatin method with interval bound propagation (IBP).\n\n* [Evaluating Robustness of Neural Networks with Mixed Integer Programming](https://arxiv.org/abs/1711.07356) (ICLR 2019) \u003cbr/\u003e This paper use mixed integer programming (MIP) method to solve the verification problem.\n\n* [Efficient Neural Network Robustness Certification with General Activation Functions](https://arxiv.org/abs/1811.00866) (NeurIPS 2018) \u003cbr/\u003e This paper proposes the verification method CROWN for general activation with locally linear or quadratic approximation.\n\n* [A Unified View of Piecewise Linear Neural Network Verification](https://arxiv.org/abs/1711.00455) (NeurIPS 2018) \u003cbr/\u003e This paper presents a unified framework and an empirical benchmark on previous verification methods\n\n* [Scaling Provable Adversarial Defenses](http://papers.nips.cc/paper/8060-scaling-provable-adversarial-defenses.pdf) (NeurIPS 2018) \u003cbr/\u003e They add three tricks to improve the scalability (to CIFAR-10) of previously proposed method in ICML.\n\n* [Provable Defenses against Adversarial Examples via the Convex Outer Adversarial Polytope](https://arxiv.org/pdf/1711.00851.pdf) (ICML 2018) \u003cbr/\u003e By robust optimization (via a linear program), they can get a point-wise bound of robustness, where no adversarial example exists in the bound. Experiments are done on MNIST.\n\n* [Towards Fast Computation of Certified Robustness for ReLU Networks](https://arxiv.org/abs/1804.09699) (ICML 2018) \u003cbr/\u003e This paper proposes the Fast-Lin and Fast-Lip methods.\n\n* [Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach](https://arxiv.org/abs/1801.10578) (ICLR 2018) \u003cbr/\u003e This paper proposes the CLEVER method to estimate the upper bound of specification.\n\n* [Certified Defenses against Adversarial Examples](https://arxiv.org/abs/1801.09344) (ICLR 2018) \u003cbr/\u003e This paper proposes the certified training with semidefinite relaxation.\n\n* [A Dual Approach to Scalable Verification of Deep Networks](https://arxiv.org/abs/1803.06567) (UAI 2018) \u003cbr/\u003e This paper solves the dual problem to provide an upper bound of the primary specification problem for verification.\n\n* [Reluplex: An efficient SMT solver for verifying deep neural networks](https://arxiv.org/pdf/1702.01135.pdf) (CAV 2017) \u003cbr/\u003e This paper use satisfiability modulo theory (SMT) solvers for the verification problem.\n\n* [Automated Verification of Neural Networks: Advances, Challenges and Perspectives](https://arxiv.org/pdf/1805.09938.pdf) \u003cbr/\u003e This paper provides an overview of main verification methods, and introduces previous work on combining automated verification with machine learning. They also give some insights on future tendency of the combination between these two domains.\n\n\u003ca id='Theoretical'\u003e\u003c/a\u003e\n## Theoretical Analysis\n* [Towards Deep Learning Models Resistant to Large Perturbations](https://arxiv.org/pdf/2003.13370.pdf) \u003cbr/\u003e This paper prove that the weight initialization of a already robust model on small perturbation can be helpful for training on large perturbations.\n\n* [Improved Sample Complexities for Deep Neural Networks and Robust Classification via an All-Layer Margin](https://openreview.net/forum?id=HJe_yR4Fwr) (ICLR 2020) \u003cbr/\u003e This paper connect the generalization gap w.r.t all-layer margin, and propose a variant of adversarial training, where the perturbations can be imposed on each layer in network.\n\n* [Adversarial Examples Are Not Bugs, They Are Features](https://arxiv.org/pdf/1905.02175.pdf) (NeurIPS 2019) \u003cbr/\u003e They claim that adversarial examples can be directly attributed to the presence of non-robust features, which are highly predictive but locally quite sensitive.\n\n* [First-order Adversarial Vulnerability of Neural Networks and Input Dimension](https://arxiv.org/pdf/1802.01421.pdf) (ICML 2019) \u003cbr/\u003e This paper demonsrate the relations among adversarial vulnerability and gradient norm and input dimension with comprehensive empirical experiments.\n\n* [Adversarial Examples from Computational Constraints](https://arxiv.org/pdf/1805.10204.pdf) (ICML 2019) \u003cbr/\u003e The authors argue that the exsitence of adversarial examples could stem from computational constrations.\n\n* [Adversarial Examples Are a Natural Consequence of Test Error in Noise](https://arxiv.org/pdf/1901.10513.pdf) (ICML 2019) \u003cbr/\u003e This paper connects the relation between the general corruption robustness and the adversarial robustness, and recommand the adversarial defenses methods to be also tested on general-purpose noises.\n\n* [PAC-learning in the presence of evasion adversaries](https://arxiv.org/pdf/1806.01471.pdf) (NeurIPS 2018) \u003cbr/\u003e The authors analyze the adversarial attacks from the PAC-learning framework.\n\n* [Adversarial Vulnerability for Any Classifier](http://papers.nips.cc/paper/7394-adversarial-vulnerability-for-any-classifier.pdf) (NeurIPS 2018) \u003cbr/\u003e Uniform upper bound of robustness for any classifier on the data sampled from smooth genertive models.\n\n* [Adversarially Robust Generalization Requires More Data](http://papers.nips.cc/paper/7749-adversarially-robust-generalization-requires-more-data.pdf) (NeurIPS 2018) \u003cbr/\u003e This paper show that robust generalization requires much more sample complexity compared to standard generlization on two simple data distributional models. \n\n* [Robustness of Classifiers:from Adversarial to Random Noise](http://papers.nips.cc/paper/6331-robustness-of-classifiers-from-adversarial-to-random-noise.pdf) (NeurIPS 2016)\n\n\u003ca id='Empirical'\u003e\u003c/a\u003e\n## Empirical Analysis\n* [Aliasing and adversarial robust generalization of CNNs](https://link.springer.com/article/10.1007/s10994-022-06222-8) (ECML 2022) This paper empirically demonstrates that adversarial robust models learn to downsample more accurate and thus suffer significantly less from downsampling artifacts, aka. aliasing, than simple non-robust baseline models.\n\n* [Adversarial Robustness Through the Lens of Convolutional Filters](https://openaccess.thecvf.com/content/CVPR2022W/ArtOfRobust/html/Gavrikov_Adversarial_Robustness_Through_the_Lens_of_Convolutional_Filters_CVPRW_2022_paper.html) (CVPR-W 2022) \u003cbr/\u003e This paper compares the learned convolution filters of a large amount of pretrained robust models against identical networks trained without adversarial defenses. The authors show that robust models form more orthogonal, diverse, and less sparse convolution filters, but differences diminish with increasing dataset complexity.\n\n* [CNN Filter DB: An Empirical Investigation of Trained Convolutional Filters](https://openaccess.thecvf.com/content/CVPR2022/html/Gavrikov_CNN_Filter_DB_An_Empirical_Investigation_of_Trained_Convolutional_Filters_CVPR_2022_paper.html)  (CVPR 2022) \u003cbr/\u003e This paper performs an empirical analysis of learned 3x3 convolution filters in various CNNs and shows that robust models learn less sparse and more diverse convolution filters.\n\n* [PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures](https://arxiv.org/abs/2112.05135) (CVPR 2022) \u003cbr/\u003e This paper uses dreamlike pictures as data augmentation to generally improve robustness (remove texture-based confounders).\n\n* [How Benign is Benign Overfitting](https://openreview.net/pdf?id=g-wu9TMPODo) (ICLR 2021) \u003cbr/\u003e This paper shows that adversarial vulnerability may come from bad\ndata and (poorly) trained models, namely, learned representations.\n\n* [Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples](https://arxiv.org/abs/2010.03593) \u003cbr/\u003e This paper explores the limits of adversarial training on CIFAR-10 by applying large model architecture, weight moving average, smooth activation and more training data to achieve SOTA robustness under norm-bounded constraints.\n\n* [Bag of Tricks for Adversarial Training](https://openreview.net/forum?id=Xb8xvrtB8Ce) (ICLR 2021) \u003cbr/\u003e This paper provides an empirical study on the usually overlooked hyperparameters used in adversarial training, and show that inappropriate settings can largely affect the performance of adversarially trained models.\n\n* [Neural Anisotropy Directions](https://arxiv.org/pdf/2006.09717.pdf) (NeurIPS 2020) \u003cbr/\u003e This paper shows that there exist directional inductive biases of model architectures, which can explain the model reaction against certain adversarial perturbation.\n\n* [Hold me tight! Influence of discriminative features on deep network boundaries](https://arxiv.org/abs/2002.06349) (NeurIPS 2020) \u003cbr/\u003e This paper empirically shows that decision boundaries are constructed along discriminative features, and explain the mechanism of adversarial training.\n\n* [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://arxiv.org/abs/2003.01690) (ICML 2020) \u003cbr/\u003e An comprehensive empirical evaluations on some of the existing defense methods. \n\n* [Attacks Which Do Not Kill Training Make Adversarial Learning Stronger](https://arxiv.org/pdf/2002.11242.pdf) (ICML 2020) \u003cbr/\u003e This paper also advovate for early-stop during adversarial training.\n\n* [Overfitting in adversarially robust deep learning](https://arxiv.org/pdf/2002.11569.pdf) (ICML 2020) \u003cbr/\u003e This paper shows the phenomena of overfitting when training robust models with sufficient empirical experiments (codes provided in paper).\n\n* [When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks](https://arxiv.org/abs/1911.10695) \u003cbr/\u003e This paper leverages NAS to understand the influence of network architectures against adversarial attacks. It reveals several useful observations on designing robust network architectures.\n\n* [Adversarial Examples Improve Image Recognition](https://arxiv.org/pdf/1911.09665.pdf) \u003cbr/\u003e This paper shows that an auxiliary BN for adversarial examples can improve generalization performance.\n\n* [Intriguing Properties of Adversarial Training at Scale](https://openreview.net/forum?id=HyxJhCEFDS\u0026noteId=rJxeamAAKB) (ICLR 2020) \u003cbr/\u003e This paper investigates the effects of BN and deeper models for adversarial training on ImageNet.\n\n* [A Fourier Perspective on Model Robustness in Computer Vision](https://papers.nips.cc/paper/9483-a-fourier-perspective-on-model-robustness-in-computer-vision.pdf) (NeurIPS 2019) \u003cbr/\u003e This paper analyzes different types of noises (including adversarial ones) from the Fourier perspective, and observes some relationship between the robustness and the Fourier frequency. \n\n* [Interpreting Adversarially Trained Convolutional Neural Networks](https://arxiv.org/pdf/1905.09797.pdf) (ICML 2019) \u003cbr/\u003e This paper show that adversarial trained models can alleviate the texture bias and learn a more shape-biased representation.\n\n* [On Evaluating Adversarial Robustness](https://arxiv.org/pdf/1902.06705.pdf) \u003cbr/\u003e Some analyses on how to correctly evaluate the robustness of adversarial defenses.\n\n* [Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models](https://openaccess.thecvf.com/content_ECCV_2018/html/Dong_Su_Is_Robustness_the_ECCV_2018_paper.html) \u003cbr/\u003e This paper empirically studies the effects of model architectures (trained on ImageNet) on robustness and accuracy.\n\n* [Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong](https://arxiv.org/pdf/1706.04701.pdf) \u003cbr/\u003e This paper tests some ensemble of existing detection-based defenses, and claim that these ensemble defenses could still be evaded by white-box attacks.\n\n\u003ca id='Beyond_Safety'\u003e\u003c/a\u003e\n## Beyond Safety\n* [Robust Models are less Over-Confident](https://openreview.net/forum?id=5K3uopkizS) (NeurIPS 2022) \u003cbr/\u003e This paper analyzes the (over)confidence of robust CNNs and concludes that robust models that are significantly less overconfident with their decisions, even on clean data. Further, the authors provide a model zoo of various CNNs trained with and without adversarial defenses.\n\n* [Improved Autoregressive Modeling with Distribution Smoothing](https://openreview.net/forum?id=rJA5Pz7lHKb) (ICLR 2021) \u003cbr/\u003e This paper apply similar idea of randomized smoothing into autoregressive generative modeling, which first modeling a smoothed data distribution and then denoise the sampled data.\n\n* [Defending Against Image Corruptions Through Adversarial Augmentations](https://arxiv.org/pdf/2104.01086.pdf) \u003cbr/\u003e This paper proposes AdversarialAugment method to adversarially craft corrupted augmented images during training.\n\n* [On the effectiveness of adversarial training against common corruptions](https://arxiv.org/pdf/2103.02325.pdf) \u003cbr/\u003e This paper studies how to use adversarial training (both Lp and a relaxation of perceptual adversarial training) to improve the performance on common image corruptions (CIFAR-10-C / ImageNet-100-C).\n\n\n* [Unadversarial Examples: Designing Objects for Robust Vision](https://arxiv.org/pdf/2012.12235.pdf) (NeurIPS 2021) \u003cbr/\u003e This paper turns the weakness of adversarial examples into strength, and proposes to use unadversarial examples to enhance model performance and robustness.\n\n* [Self-supervised Learning with Adversarial Training](https://github.com/P2333/Papers-of-Robust-ML) ([1](https://proceedings.neurips.cc/paper/2020/hash/1f1baa5b8edac74eb4eaa329f14a0361-Abstract.html), [2](https://proceedings.neurips.cc/paper/2020/hash/c68c9c8258ea7d85472dd6fd0015f047-Abstract.html), [3](https://proceedings.neurips.cc/paper/2020/hash/ba7e36c43aff315c00ec2b8625e3b719-Abstract.html)) (NeurIPS 2020) \u003cbr/\u003e These three papers work on embedding adversarial training mechanism into contrastive-based self-supervised learning. They show that AT mechanism can promote the learned representations.\n\n* [Do Adversarially Robust ImageNet Models Transfer Better?](https://proceedings.neurips.cc/paper/2020/hash/24357dd085d2c4b1a88a7e0692e60294-Abstract.html) (NeurIPS 2020) \u003cbr/\u003e This paper show that an adversarially robust model can work better for transfer learning, which encourage the learning process to focus on semantic features.\n\n* [Adversarial Examples Improve Image Recognition](https://cs.jhu.edu/~alanlab/Pubs20/xie2020adversarial.pdf) (CVPR 2020) \u003cbr/\u003e This paper treat adversarial training as a regularization strategy for traditional classification task, and achieve SOTA clean performance on ImageNet without extra data.\n\n\u003ca id='Seminal_work'\u003e\u003c/a\u003e\n## Seminal Work\n* [Unsolved Problems in ML Safety](https://arxiv.org/pdf/2109.13916.pdf) \u003cbr/\u003e A comprehensive roadmap for future researches in Trustworthy ML. \n\n* [Towards Deep Learning Models Resistant to Adversarial Attacks](https://arxiv.org/pdf/1706.06083.pdf) (ICLR 2018) \u003cbr/\u003e This paper proposed projected gradient descent (PGD) attack, and the PGD-based adversarial training.\n\n* [Adversarial examples are not easily detected: Bypassing ten detection methods](https://dl.acm.org/citation.cfm?Id=3140444) (AISec 17) \u003cbr/\u003e This paper first desgined different adaptive attacks for detection-based methods.\n\n* [Explaining and Harnessing Adversarial Examples](https://arxiv.org/abs/1412.6572) (ICLR 2015) \u003cbr/\u003e This paper proposed fast gradient sign method (FGSM), and the framework of adversarial training.\n\n* [Intriguing properties of neural networks](https://arxiv.org/abs/1312.6199) (ICLR 2014) \u003cbr/\u003e This paper first introduced the concept of adversarial examples in deep learning, and provided a L-BFGS based attack method.\n\n\u003ca id='Benchmark_Datasets'\u003e\u003c/a\u003e\n## Benchmark Datasets\n* [RobustBench: a standardized adversarial robustness benchmark](https://arxiv.org/pdf/2010.09670.pdf) \u003cbr/\u003e A standardized robustness benchmark with 50+ models together with the [Model Zoo](https://github.com/RobustBench/robustbench). \n\n* [Natural adversarial examples](https://arxiv.org/pdf/1907.07174.pdf) \u003cbr/\u003e ImageNet-A dataset.\n\n* [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://arxiv.org/pdf/1903.12261.pdf) (ICLR 2019) \u003cbr/\u003e ImageNet-C dataset.\n\n* [Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness](https://arxiv.org/pdf/1811.12231.pdf) (ICLR 2018) \u003cbr/\u003e This paper empirically demonstrate that shape-based features lead to more robust models. They also provide the Styled-ImageNet dataset.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FP2333%2FPapers-of-Robust-ML","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FP2333%2FPapers-of-Robust-ML","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FP2333%2FPapers-of-Robust-ML/lists"}