{"id":20663728,"url":"https://github.com/vita-group/augmax","last_synced_at":"2025-04-19T15:56:03.697Z","repository":{"id":107045068,"uuid":"411432327","full_name":"VITA-Group/AugMax","owner":"VITA-Group","description":"[NeurIPS'21] \"AugMax: Adversarial Composition of Random Augmentations for Robust Training\" by Haotao Wang, Chaowei Xiao, Jean Kossaifi, Zhiding Yu, Animashree Anandkumar, and Zhangyang Wang.","archived":false,"fork":false,"pushed_at":"2021-12-29T08:00:24.000Z","size":937,"stargazers_count":125,"open_issues_count":0,"forks_count":21,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-03-29T09:42:03.653Z","etag":null,"topics":["adversarial-learning","imagenet-c","model-robustness","normalization-techniques","out-of-distribution"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VITA-Group.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-28T20:40:58.000Z","updated_at":"2024-11-19T18:11:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"115dbe05-00d4-48e3-8be2-ed621f18bb2c","html_url":"https://github.com/VITA-Group/AugMax","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FAugMax","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FAugMax/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FAugMax/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FAugMax/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VITA-Group","download_url":"https://codeload.github.com/VITA-Group/AugMax/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249731218,"owners_count":21317341,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-learning","imagenet-c","model-robustness","normalization-techniques","out-of-distribution"],"created_at":"2024-11-16T19:19:31.859Z","updated_at":"2025-04-19T15:56:03.691Z","avatar_url":"https://github.com/VITA-Group.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AugMax: Adversarial Composition of Random Augmentations for Robust Training\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)\n\nHaotao Wang, Chaowei Xiao, Jean Kossaifi, Zhiding Yu, Anima Anandkumar, and Zhangyang Wang\n\nIn _NeurIPS_ 2021\n\n## Overview\n\nWe propose AugMax, a data augmentation framework to unify the diversity and hardness. Being a stronger form of data augmentation, AugMax leads to a significantly augmented input distribution which makes model training more challenging. To solve this problem, we further design a disentangled normalization module, termed DuBIN (Dual-Batch-and-Instance Normalization) that disentangles the instance-wise feature heterogeneity arising from AugMax. AugMax-DuBIN leads to significantly improved out-of-distribution robustness, outperforming prior arts by 3.03%, 3.49%, 1.82% and 0.71% on CIFAR10-C, CIFAR100-C, Tiny ImageNet-C and ImageNet-C.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/AugMax.PNG\" alt=\"AugMax\" width=\"800\"/\u003e\u003c/br\u003e\n  \u003cspan align=\"center\"\u003eAugMax achieves a unification between hard and diverse training samples.\u003c/span\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/results.PNG\" alt=\"results\" width=\"800\"/\u003e\u003c/br\u003e\n  \u003cspan align=\"center\"\u003eAugMax achieves state-fo-the-art performance on CIFAR10-C, CIFAR100-C, Tiny ImageNet-C and ImageNet-C.\u003c/span\u003e\n\u003c/p\u003e\n\n\n## Training\n\nAssume all datasets are stored in `\u003cdata_root_path\u003e`. For example, CIFAR-10 is in `\u003cdata_root_path\u003e/cifar-10-batches-py/` and ImageNet training set is in `\u003cdata_root_path\u003e/imagenet/train`. \n\nAugMax-DuBIN training on `\u003cdataset\u003e` with `\u003cbackbone\u003e` (The outputs will be saved in `\u003csave_root_path\u003e`):\n\n```\npython augmax_training_ddp.py --gpu 0 --srp \u003cwhere_you_save_the_outputs\u003e --drp \u003cdata_root_path\u003e --ds \u003cdataset\u003e --md \u003cbackbone\u003e --Lambda \u003clambda_value\u003e --steps \u003cinner_max_step_number\u003e\n```\n\nFor example:\n\nAugMax-DuBIN on CIFAR10 with ResNeXt29 (By default we use `Lambda=10` on CIFAR10/100 and Tiny ImageNet.):\n\n```\npython augmax_training_ddp.py --gpu 0 --drp /ssd1/haotao/datasets --ds cifar10 --md ResNeXt29 --Lambda 10 --steps 10\n```\n\nAugMax-DuBIN on CIFAR100 with ResNet18 (We use `Lambda=1` instead of `Lambda=10` in this particular experiment, as noted in the paper.):\n\n```\npython augmax_training_ddp.py --gpu 0 --drp /ssd1/haotao/datasets --ds cifar100 --md ResNet18 --Lambda 1 --steps 10\n```\n\nAugMax-DuBIN on ImageNet with ResNet18 (By default we use `Lambda=12` on ImageNet. On ImageNet, **weight decay `wd=1e-4`** instead of the default value in the code, which is for CIFAR-level datasets.):\n\n```\nNCCL_P2P_DISABLE=1 python augmax_training_ddp.py --gpu 0 --drp /ssd1/haotao/datasets --ds IN --md ResNet18 --Lambda 12 -e 90 --wd 1e-4 --decay multisteps --de 30 60 --ddp --dist_url tcp://localhost:23456\n```\n\nAugMax-DuBIN + DeepAug on ImageNet with ResNet18:\n\n```\nNCCL_P2P_DISABLE=1 python augmax_training_ddp.py --gpu 0 --drp /ssd1/haotao/datasets --ds IN --md ResNet18 --deepaug --Lambda 12 -e 30 --wd 1e-4 --decay multisteps --de 10 20 --ddp --dist_url tcp://localhost:23456\n```\n\n## Pretrained models\n\nThe pretrained models are available on [Google Drive](https://drive.google.com/drive/folders/1GH1fjWQuTYruUU7P7BM52Erg2tAfNJuj?usp=sharing).\n\n## Testing\n\nTo test the model trained on `\u003cdataset\u003e` with `\u003cbackbone\u003e` and saved to `\u003cckpt_path\u003e`:\n\n```\npython test.py --gpu 0 --ds \u003cdataset\u003e --drp \u003cdata_root_path\u003e --md \u003cbackbone\u003e --mode all --ckpt_path \u003cckpt_path\u003e\n```\n\nFor example:\n\n```\npython test.py --gpu 0 --ds cifar10 --drp /ssd1/haotao/datasets --md ResNet18_DuBIN --mode all --ckpt_path augmax_training/cifar10/ResNet18_DuBIN/fat-1-untargeted-5-0.1_Lambda10.0_e200-b256_sgd-lr0.1-m0.9-wd0.0005_cos\n```\n\n## Citation\n```\n@inproceedings{wang2021augmax,\n  title={AugMax: Adversarial Composition of Random Augmentations for Robust Training},\n  author={Wang, Haotao and Xiao, Chaowei and Kossaifi, Jean and Yu, Zhiding and Anandkumar, Anima and Wang, Zhangyang},\n  booktitle={NeurIPS},\n  year={2021}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvita-group%2Faugmax","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvita-group%2Faugmax","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvita-group%2Faugmax/lists"}