{"id":20663791,"url":"https://github.com/vita-group/atmc","last_synced_at":"2025-08-07T21:16:57.801Z","repository":{"id":54317324,"uuid":"212119448","full_name":"VITA-Group/ATMC","owner":"VITA-Group","description":"[NeurIPS'2019] Shupeng Gui, Haotao Wang, Haichuan Yang, Chen Yu, Zhangyang Wang, Ji Liu, “Model Compression with Adversarial Robustness: A Unified Optimization Framework”","archived":false,"fork":false,"pushed_at":"2021-12-30T16:30:16.000Z","size":53054,"stargazers_count":50,"open_issues_count":2,"forks_count":10,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-07-25T08:59:51.288Z","etag":null,"topics":["model-compression","pruning","quantization","robustness","unified-optimization-framework"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VITA-Group.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-01T14:36:45.000Z","updated_at":"2025-05-20T11:56:00.000Z","dependencies_parsed_at":"2022-08-13T11:50:44.563Z","dependency_job_id":null,"html_url":"https://github.com/VITA-Group/ATMC","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/VITA-Group/ATMC","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FATMC","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FATMC/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FATMC/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FATMC/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VITA-Group","download_url":"https://codeload.github.com/VITA-Group/ATMC/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FATMC/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269325198,"owners_count":24398147,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-07T02:00:09.698Z","response_time":73,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["model-compression","pruning","quantization","robustness","unified-optimization-framework"],"created_at":"2024-11-16T19:19:52.879Z","updated_at":"2025-08-07T21:16:57.755Z","avatar_url":"https://github.com/VITA-Group.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Model Compression with Adversarial Robustness: A Unified Optimization Framework (ATMC)\n### Authors:\n* [Shupeng Gui](https://sites.google.com/view/shupeng-gui/home)*\n* [Haotao Wang](http://people.tamu.edu/~htwang/)*\n* [Haichuan Yang](https://www.cs.rochester.edu/u/hyang36/)\n* [Chen Yu](https://www.linkedin.com/in/lukecyu/en)\n* [Zhangyang Wang](https://www.atlaswang.com/)\n* [Ji Liu](https://scholar.google.com/citations?user=RRzVwKkAAAAJ\u0026hl=en)\n\n*: Equal Contribution\n\n### Overview\nIn this repo, we present one example implementation of ATMC robust learning framework from NeurIPS 2019 paper [Model Compression with Adversarial Robustness: A Unified Optimization Framework](https://arxiv.org/abs/1902.03538).\n\nWe propose a noval *Adversarially Trained Model Compression* (ATMC) framework, which conducts a unified constrained optimization formulation, where existing compression means (pruning, factorization, quantization) are all integrated into the constraints. An extensive group of experiments are presented, demonstrating that ATMC obtains remarkably more favorable trade-off among model size, accuracy and robustness, over currently available alternatives in various settings.\n![ATMC Experiments Results on Various Compression Ratio](ATMC_exps.png)\n\n### Requirements\nAll experiments were executed on a Linux machine with Intel I7-6700k, 64 GB memory and two GTX1080 Graphics Card. To reproduce the experiment results in the paper, some experiment parameter settings could be tuned for the user case (such as batch size).\n\nThe software environment bases on Pytorch (\u003e=1.0.0).\n\n### Experiment Example\nOne example shows how to setup an experiment on CIFAR-10 dataset.\n\nFirst of all, we need to obtain a dense model for successors.\n```\npython cifar/train_proj_admm_quant.py \\\n--raw_train \\\n--epochs 200 \\\n--lr 0.05 \\\n--decreasing_lr 80,120,150 \\\n--gpu 0 \\\n--savedir log/resnet/pretrain \\          \n--data_root [cifar/data/dir] \\            \n--attack_algo pgd \\\n--attack_eps 4 \\\n--defend_algo pgd \\\n--defend_eps 4 \\\n--defend_iter 7 \\\n--save_model_name cifar10_res_pgd_raw.pth \\\n--quantize_bits 32 \\\n--prune_ratio 1.0\n```\nThe dense model will be stored in 'log/resnet/pretrain'. Then, the second round execution of the python script will be operated with\n```\npython cifar/train_proj_admm_quant.py \\\n--epochs 200 \\\n--lr 0.01 \\\n--decreasing_lr 30,60,90,120 \\\n--gpu 0 \\\n--savedir log/resnet/l0 \\\n--loaddir log/resnet/pretrain \\\n--model_name cifar10_res_pgd_raw.pth \\  \n--data_root [cifar/data/dir] \\       \n--quantize_bits 32 \\\n--attack_algo pgd \\\n--attack_eps 4 \\\n--defend_algo pgd \\                                             \n--defend_eps 4 \\\n--defend_iter 7 \\\n--save_model_name cifar10_resnet_pgd_4_l0proj_0.005.pth \\\n--quantize_bits 32 \\ \n--prune_algo l0proj \\\n--prune_ratio 0.005 \\\n```\nAfter the process above, we get an sparse model with 0.005 compression ratio. Then we operate the ATMC process based on this pre-trained model.\n```\npython cifar/train_proj_admm_quant.py \\\n--epochs 200 \\                     \n--lr 0.005 \\     \n--decreasing_lr 60,80,120 \\   \n--gpu 0 \\                          \n--savedir log/resnet/atmc \\                                             \n--loaddir log/resnet/l0 \\                                               \n--save_model_name cifar10_resnet_pgd_4_atmc_0.005_32bit.pth \\ \n--data_root [cifar/data/dir] \\     \n--attack_algo pgd \\\n--attack_eps 4 \\\n--defend_algo pgd \\                                             \n--defend_eps 4 \\    \n--defend_iter 7 \\                                              \n--prune_algo l0proj \\\n--abc_special \\        \n--prune_ratio 0.005 \\ \n--quantize_bits 32 \\\n--defend_iter 7 \\\n--model_name sparse_cifar10_resnet_pgd_4_l0proj_0.005.pth\n```\nIf we want to apply the unified pruning and quantization strategy, we are going to run\n```\npython cifar100/train_proj_admm_quant.py \\\n--epochs 150 \\\n--lr 0.005 \\\n--decreasing_lr 60,80,120 \\\n--gpu 0 \\\n--savedir log/resnet/atmc8bit \\\n--loaddir log/resnet/atmc \\\n--save_model_name cifar10_pgd_4_atmc_0.005_8bit.pth \\\n--data_root [cifar/data/dir] \\\n--attack_algo pgd \\\n--attack_eps 4 \\\n--abc_special \\\n--abc_initialize \\\n--defend_algo pgd \\\n--defend_eps 4 \\\n--defend_iter 7 \\\n--quantize_algo kmeans_nnz_fixed_0_center \\\n--quant_interval 10 \\\n--prune_algo l0proj \\\n--prune_ratio 0.005 \\\n--quantize_bits 8 \\\n--model_name sparse_cifar10_pgd_4_atmc_0.005_32bit.pth\n```\n\nIf you find this repo useful, please cite:\n```\n@InProceedings{gui2019ATMC,\n  title = \t {Model Compression with Adversarial Robustness: A Unified Optimization Framework},\n  author = \t {Gui, Shupeng and Wang, Haotao and Yang, Haichuan and Yu, Chen and Wang, Zhangyang and Liu, Ji},\n  booktitle = \t {Proceedings of the 33rd Conference on Neural Information Processing Systems},\n  year = \t {2019},\n}\n```\n\n### Reference Implementation\nThanks to the reference repo [pytorch-playground](https://github.com/aaron-xichen/pytorch-playground).\n\n### Dependencies\n- pytorch (\u003e=1.0.0)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvita-group%2Fatmc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvita-group%2Fatmc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvita-group%2Fatmc/lists"}