{"id":19382873,"url":"https://github.com/locuslab/smoothing","last_synced_at":"2025-04-07T07:08:55.379Z","repository":{"id":45791664,"uuid":"169513830","full_name":"locuslab/smoothing","owner":"locuslab","description":"Provable adversarial robustness at ImageNet scale","archived":false,"fork":false,"pushed_at":"2019-05-20T04:50:04.000Z","size":7553,"stargazers_count":383,"open_issues_count":5,"forks_count":76,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-03-31T05:08:04.263Z","etag":null,"topics":["adversarial-machine-learning"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1902.02918","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/locuslab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-02-07T03:27:12.000Z","updated_at":"2025-03-12T07:26:12.000Z","dependencies_parsed_at":"2022-07-16T16:00:37.020Z","dependency_job_id":null,"html_url":"https://github.com/locuslab/smoothing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/locuslab%2Fsmoothing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/locuslab%2Fsmoothing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/locuslab%2Fsmoothing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/locuslab%2Fsmoothing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/locuslab","download_url":"https://codeload.github.com/locuslab/smoothing/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247608151,"owners_count":20965952,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-machine-learning"],"created_at":"2024-11-10T09:23:39.148Z","updated_at":"2025-04-07T07:08:55.353Z","avatar_url":"https://github.com/locuslab.png","language":"Python","funding_links":[],"categories":["Defenses \u0026 Robustness","Formal Verification \u0026 Robustness"],"sub_categories":["Certified Robustness"],"readme":"# Certified Adversarial Robustness via Randomized Smoothing\n\nThis repository contains code and trained models for the paper [Certified Adversarial Robustness via Randomized Smoothing](https://arxiv.org/abs/1902.02918) by [Jeremy Cohen](http://cs.cmu.edu/~jeremiac), Elan Rosenfeld, and [Zico Kolter](http://zicokolter.com).\n\nRandomized smoothing is a **provable** adversarial defense in L2 norm which **scales to ImageNet.**\nIt's also SOTA on the smaller datasets like CIFAR-10 and SVHN where other provable L2-robust classifiers are viable.\n\n## How does it work?\n\nFirst, you train a neural network _f_ with Gaussian data augmentation at variance \u0026sigma;\u003csup\u003e2\u003c/sup\u003e.\nThen you leverage _f_ to create a new, \"smoothed\" classifier _g_, defined as follows:\n_g(x)_ returns the class which _f_ is most likely to return when _x_\nis corrupted by isotropic Gaussian noise with variance \u0026sigma;\u003csup\u003e2\u003c/sup\u003e. \n\n\u003cp\u003e\n\u003cimg src=\"figures/panda_577.png\" height=\"224\" width=\"224\" \u003e\n\u003cimg src=\"figures/panda_0.50.gif\" height=\"224\" width=\"224\" \u003e\n\u003c/p\u003e\n\n\nFor example, let _x_ be the image above on the left.\nSuppose that when _f_ classifies _x_ corrupted by Gaussian noise (the GIF on the right), _f_ returns \"panda\"\n98\\% of the time and \"gibbon\" 2% of the time.\nThen the prediction of _g_ at _x_ is defined to be \"panda.\"\n \n\nInterestingly, _g_ is **provably** robust within an L2 norm ball around _x_, in the sense that for any perturbation \n\u0026delta; with sufficiently small L2 norm, _g(x+\u0026delta;)_ is guaranteed to be \"panda.\"\nIn this particular example, _g_ will be robust around _x_ within an L2 radius of \u0026sigma; \u0026Phi;\u003csup\u003e-1\u003c/sup\u003e(0.98) \u0026asymp; 2.05 \u0026sigma;,\nwhere \u0026Phi;\u003csup\u003e-1\u003c/sup\u003e is the inverse CDF of the standard normal distribution.\n\nIn general, suppose that when _f_ classifies noisy corruptions of _x_, the class \"panda\" is returned with probability _p_ (with _p_ \u003e 0.5).\nThen _g_ is guaranteed to classify \"panda\" within an L2 ball around _x_ of radius \u0026sigma; \u0026Phi;\u003csup\u003e-1\u003c/sup\u003e(_p_).\n\n### What's the intuition behind this bound?\n\nWe know that _f_ classifies noisy corruptions of _x_ as \"panda\" with probability 0.98.\nAn equivalent way of phrasing this that the Gaussian distribution N(x, \u0026sigma;\u003csup\u003e2\u003c/sup\u003eI) puts measure 0.98 on\nthe decision region of class \"panda,\" defined as the set {x': f(x') = \"panda\"}.\nYou can prove that no matter how the decision regions of _f_ are \"shaped\", for any \u0026delta; with\n||\u0026delta;||\u003csub\u003e2\u003c/sub\u003e \u003c \u0026sigma; \u0026Phi;\u003csup\u003e-1\u003c/sup\u003e(0.98), the translated Gaussian N(x+\u0026delta;, \u0026sigma;\u003csup\u003e2\u003c/sup\u003eI) is guaranteed to put measure \u003e 0.5 on the decision region of\nclass \"panda,\" implying that _g(x+\u0026delta;)_ = \"panda.\"\n\n### Wait a minute...\nThere's one catch: it's not possible to actually evaluate the smoothed classifer _g_.\nThis is because it's not possible to exactly compute the probability distribution over the classes when _f_'s input is corrupted by Gaussian noise.\nFor the same reason, it's not possible to exactly compute the radius in which _g_ is provably robust.\n\nInstead, we give Monte Carlo algorithms for both\n1. **prediction**: evaluating _g_(x)\n2. **certification**: computing the L2 radius in which _g_ is robust around _x_\n\nwhich are guaranteed to return a correct answer with arbitrarily high probability.\n\nThe prediction algorithm does this by abstaining from making any prediction when it's a \"close call,\" e.g. if\n 510 noisy corruptions of _x_ were classified as \"panda\" and 490 were classified as \"gibbon.\"\nPrediction is pretty cheap, since you don't need to use very many samples.\nFor example, with our ImageNet classifier, making a prediction using 1000 samples took 1.5 seconds, and our classifier abstained 3\\% of the time.\n\nOn the other hand, certification is pretty slow, since you need _a lot_ of samples to say with high\nprobability that the measure under N(x, \u0026sigma;\u003csup\u003e2\u003c/sup\u003eI) of the \"panda\" decision region is close to 1.\nIn our experiments we used 100,000 samples, so making each certification took 150 seconds.\n\n### Related work\n\nRandomized smoothing was first proposed in [Certified Robustness to Adversarial Examples with Differential Privacy](https://arxiv.org/abs/1802.03471)\nand later improved upon in [Second-Order Adversarial Attack and Certified Robustness](https://arxiv.org/abs/1809.03113).\nWe simply tightened the analysis and showed that it outperforms the other provably L2-robust classifiers that have been proposed in the literature. \n\n## ImageNet results\n\nWe constructed three randomized smoothing classifiers for ImageNet, with the hyperparameter\n\u0026sigma; set to 0.25, 0.50, and 1.00.\nHere's what the panda image looks like under these three noise levels:\n\n\u003cp\u003e\n\u003cimg src=\"figures/panda_0.25.gif\" height=\"224\" width=\"224\" \u003e\n\u003cimg src=\"figures/panda_0.50.gif\" height=\"224\" width=\"224\" \u003e\n\u003cimg src=\"figures/panda_1.00.gif\" height=\"224\" width=\"224\" \u003e\n\u003c/p\u003e\n\n\nThe plot below shows the certified top-1 accuracy at various radii of these three classifiers.\nThe \"certified accuracy\" of a classifier _g_ at radius _r_ is defined as test set accuracy that _g_ will \nprovably attain under any possible adversarial attack with L2 norm less than _r_. \nAs you can see, the hyperparameter \u0026sigma; controls a robustness/accuracy tradeoff: when\n\u0026sigma; is high, the standard accuracy is lower, but the classifier's correct predictions are \nrobust within larger radii.\n\n\u003ca href=\"analysis/plots/vary_noise_imagenet.pdf\"\u003e\u003cimg src=\"analysis/plots/vary_noise_imagenet.png\" height=\"300\" width=\"400\" \u003e\u003c/a\u003e\n\nTo put these numbers in context: on ImageNet, random guessing would achieve a top-1 accuracy of 0.001.\nA perturbation with L2 norm of 1.0 could change one pixel by 255, ten pixels by 80, 100 pixels by 25, or 1000 pixels by 8.  \n\n\nHere's the same data in tabular form.\nThe best \u0026sigma; for each radius is denoted with an asterisk.\n\n|  | r = 0.0 |r = 0.5 |r = 1.0 |r = 1.5 |r = 2.0 |r = 2.5 |r = 3.0 |\n| --- |  --- | --- | --- | --- | --- | --- | --- |\n\u003cb\u003e \u0026sigma; = 0.25 \u003c/b\u003e| 0.67\u003cb\u003e*\u003c/b\u003e |0.49\u003cb\u003e*\u003c/b\u003e |0.00 |0.00 |0.00 |0.00 |0.00 |\n\u003cb\u003e \u0026sigma; = 0.50 \u003c/b\u003e| 0.57 |0.46 |0.38\u003cb\u003e*\u003c/b\u003e |0.28\u003cb\u003e*\u003c/b\u003e |0.00 |0.00 |0.00 |\n\u003cb\u003e \u0026sigma; = 1.00 \u003c/b\u003e| 0.44 |0.38 |0.33 |0.26 |0.19\u003cb\u003e*\u003c/b\u003e |0.15\u003cb\u003e*\u003c/b\u003e |0.12\u003cb\u003e*\u003c/b\u003e |\n\n\n\u003c!--\n## How does it do on CIFAR-10?\n\n\u003ca href=\"analysis/plots/vary_noise_cifar10_resnet110_test.pdf\"\u003e\u003cimg src=\"analysis/plots/vary_noise_cifar10_resnet110_test.png\" height=\"300\" width=\"400\" \u003e\u003c/a\u003e\n\n\n|  | r = 0.0 |r = 0.25 |r = 0.5 |r = 0.75 |r = 1.0 |r = 1.25 |r = 1.5 |\n| --- |  --- | --- | --- | --- | --- | --- | --- |\n\u003cb\u003e \u0026sigma; = 0.25 \u003c/b\u003e| 0.77\u003cb\u003e*\u003c/b\u003e |0.60\u003cb\u003e*\u003c/b\u003e |0.42\u003cb\u003e*\u003c/b\u003e |0.24 |0.00 |0.00 |0.00 |\n\u003cb\u003e \u0026sigma; = 0.50 \u003c/b\u003e| 0.66 |0.55 |0.42 |0.31\u003cb\u003e*\u003c/b\u003e |0.21 |0.14 |0.08 |\n\u003cb\u003e \u0026sigma; = 1.00 \u003c/b\u003e| 0.47 |0.40 |0.33 |0.27 |0.22\u003cb\u003e*\u003c/b\u003e |0.18\u003cb\u003e*\u003c/b\u003e |0.14\u003cb\u003e*\u003c/b\u003e |\n--\u003e\n\n## This repository\n\n### Outline\n\nThe contents of this repository are as follows:\n\n* [code/](code) contains the code for our experiments.\n* [data/](data) contains the raw data from our experiments.\n* [analysis/](analysis) contains the plots and tables, based on the contents of [data](/data), that are shown in our paper.\n\nIf you'd like to run our code, you need to download our models from [here](https://drive.google.com/file/d/1h_TpbXm5haY5f-l4--IKylmdz6tvPoR4/view?usp=sharing)\nand then move the directory `models` into the root directory of this repo.\n\n### Smoothed classifiers\n\nRandomized smoothing is implemented in the `Smooth` class in [core.py](code/core.py).\n\n* To instantiate a smoothed clasifier _g_, use the constructor:\n\n ```def __init__(self, base_classifier: torch.nn.Module, num_classes: int, sigma: float):```\n\nwhere `base_classifier` is a PyTorch module that implements _f_, `num_classes` is the number of classes in the output\nspace, and `sigma` is the noise hyperparameter \u0026sigma; \n\n* To make a prediction at an input `x`, call:\n\n```    def predict(self, x: torch.tensor, n: int, alpha: float, batch_size: int) -\u003e int:```\n \n where `n` is the number of Monte Carlo samples and `alpha` is the confidence level.\n This function will either (1) return `-1` to abstain or (2) return a class which equals _g(x)_\n with probability at least `1 - alpha`.\n \n * To compute a radius in which _g_ is robust around an input `x`, call:\n \n ```def certify(self, x: torch.tensor, n0: int, n: int, alpha: float, batch_size: int) -\u003e (int, float):```  \n\nwhere `n0` is the number of Monte Carlo samples to use for selection (see the paper), `n` is the number of Monte Carlo\nsamples to use for estimation, and `alpha` is the confidence level.\nThis function will either return the pair `(-1, 0.0)` to abstain, or return a pair\n`(prediction, radius)`.  The probability that `certify()` will return a class not equal to _g(x)_ is no greater than `alpha`.  Another way to say this is that with probability at least `1 - alpha`, `certify()` will either abstain or return _g(x)_.\n \n### Scripts\n\n* The program [train.py](code/train.py) trains a base classifier with Gaussian data augmentation:\n\n```python code/train.py imagenet resnet50  model_output_dir --batch 400 --noise 0.50 ```  \n\nwill train a ResNet-50 on ImageNet under Gaussian data augmentation with \u0026sigma;=0.50.\n\n* The program [predict.py](code/predict.py) makes predictions using _g_ on a bunch of inputs.  For example,\n\n```python code/predict.py imagenet model_output_dir/checkpoint.pth.tar 0.50 prediction_outupt --alpha 0.001 --N 1000 --skip 100 --batch 400```\n\nwill load the base classifier saved at `model_output_dir/checkpoint.pth.tar`, smooth it using noise level \u0026sigma;=0.50,\nand classify every 100-th image from the ImageNet test set with parameters `N=1000`\nand `alpha=0.001`.\n\n* The program [certify.py](code/certify.py) certifies the robustness of _g_ on bunch of inputs.  For example,\n\n```python code/certify.py imagenet model_output_dir/checkpoint.pth.tar 0.50 certification_output --alpha 0.001 --N0 100 --N 100000 --skip 100 --batch 400```\n\nwill load the base classifier saved at `model_output_dir/checkpoint.pth.tar`, smooth it using noise level \u0026sigma;=0.50,\nand certify every 100-th image from the ImageNet test set with parameters `N0=100`, `N=100000`\nand `alpha=0.001`.\n\n* The program [visualize.py](code/visualize.py) outputs pictures of noisy examples.  For example,\n\n```python code/visualize.py imagenet visualize_output 100 0.0 0.25 0.5 1.0```\n\nwill visualize noisy corruptions of the 100-th image from the ImageNet test set with noise levels \n\u0026sigma;=0.0, \u0026sigma;=0.25, \u0026sigma;=0.50, and \u0026sigma;=1.00.   \n\n* The program [analyze.py](code/analyze.py) generates all of certified accuracy plots and tables that appeared in the\npaper.\n\nFinally, we note that [this file](experiments.MD) describes exactly how to reproduce\n our experiments from the paper.\n \n \n We're not officially releasing code for the experiments where we compared randomized smoothing against the baselines,\n since that code involved a number of hacks, but feel free to get in touch if you'd like to see that code.\n\n## Getting started\n\n1.  Clone this repository: `git clone git@github.com:locuslab/smoothing.git`\n\n2.  Install the dependencies:  \n```\nconda create -n smoothing\nconda activate smoothing\n# below is for linux, with CUDA 10; see https://pytorch.org/ for the correct command for your system\nconda install pytorch torchvision cudatoolkit=10.0 -c pytorch \nconda install scipy pandas statsmodels matplotlib seaborn\npip install setGPU\n```\n3.  Download our trained models from [here](https://drive.google.com/file/d/1h_TpbXm5haY5f-l4--IKylmdz6tvPoR4/view?usp=sharing).\n4. If you want to run ImageNet experiments, obtain a copy of ImageNet and preprocess the `val` directory to look\nlike the `train` directory by running [this script](https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh).\nFinally, set the environment variable `IMAGENET_DIR` to the directory where ImageNet is located.\n\n5. To get the hang of things, try running this command, which will certify the robustness of one of our pretrained CIFAR-10 models\non the CIFAR test set.\n```\nmodel=\"models/cifar10/resnet110/noise_0.25/checkpoint.pth.tar\"\noutput=\"???\"\npython code/certify.py cifar10 $model 0.25 $output --skip 20 --batch 400\n```\nwhere `???` is your desired output file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flocuslab%2Fsmoothing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flocuslab%2Fsmoothing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flocuslab%2Fsmoothing/lists"}