{"id":13738111,"url":"https://github.com/AIoT-MLSys-Lab/DeepAA","last_synced_at":"2025-05-08T15:32:28.518Z","repository":{"id":83645384,"uuid":"458797670","full_name":"AIoT-MLSys-Lab/DeepAA","owner":"AIoT-MLSys-Lab","description":"[ICLR 2022] \"Deep AutoAugment\" by Yu Zheng, Zhi Zhang,  Shen Yan, Mi Zhang","archived":false,"fork":false,"pushed_at":"2024-09-20T17:51:37.000Z","size":1061,"stargazers_count":62,"open_issues_count":0,"forks_count":3,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-01-08T05:01:48.327Z","etag":null,"topics":["automl","data-augmentation","deep-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AIoT-MLSys-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-02-13T12:06:34.000Z","updated_at":"2024-11-21T08:15:56.000Z","dependencies_parsed_at":"2023-04-26T21:01:52.778Z","dependency_job_id":null,"html_url":"https://github.com/AIoT-MLSys-Lab/DeepAA","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AIoT-MLSys-Lab%2FDeepAA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AIoT-MLSys-Lab%2FDeepAA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AIoT-MLSys-Lab%2FDeepAA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AIoT-MLSys-Lab%2FDeepAA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AIoT-MLSys-Lab","download_url":"https://codeload.github.com/AIoT-MLSys-Lab/DeepAA/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253096429,"owners_count":21853601,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automl","data-augmentation","deep-learning"],"created_at":"2024-08-03T03:02:11.584Z","updated_at":"2025-05-08T15:32:26.460Z","avatar_url":"https://github.com/AIoT-MLSys-Lab.png","language":"Python","readme":"# Deep AutoAugment\n\nThis is the official implementation of Deep AutoAugment ([DeepAA](https://openreview.net/forum?id=St-53J9ZARf)), a fully automated data augmentation policy search method. Leaderboard is here: https://paperswithcode.com/paper/deep-autoaugment-1\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/DeepAA.png\" alt=\"DeepAA\" width=\"70%\"\u003e \u003cbr /\u003e\n\u003c/p\u003e\n\n## 5-Minute Explanation Video\nClick the figure to watch this short video explaining our work.\n\n[![slideslive_link](./images/DeepAA_slideslive.png)](https://recorder-v3.slideslive.com/#/share?share=64177\u0026s=6d93977f-2a40-436d-a404-8808aee650fa)\n\n## Requirements\nDeepAA is implemented using TensorFlow. \nTo be consistent with previous work, we run the policy evaluation based on [TrivialAugment](https://github.com/automl/trivialaugment), which is implemented using PyTorch.\n\n### Install required packages\na. Create a conda virtual environment.\n```shell\nconda create -n deepaa python=3.7\nconda activate deepaa\n```\n\nb. Install Tensorflow and PyTorch.\n```shell\nconda install tensorflow-gpu=2.5 cudnn=8.1 cudatoolkit=11.2 -c conda-forge\npip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html\n```\n\nc. Install other dependencies.\n```shell\npip install -r requirements.txt\n```\n\n\n## Experiments\n\n### Run augmentation policy search on CIFAR-10/100. \n```shell\nexport CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7\npython DeepAA_search.py --dataset cifar10 --n_classes 10 --use_model WRN_40_2 --n_policies 6 --search_bno 1024 --pretrain_lr 0.1 --seed 1 --batch_size 128 --test_batch_size 512 --policy_lr 0.025 --l_mags 13 --use_pool --pretrain_size 5000 --nb_epochs 45 --EXP_G 16 --EXP_gT_factor=4 --train_same_labels 16\n```\n\n### Run augmentation policy search on ImageNet.\n```shell\nmkdir pretrained_imagenet\n```\nDownload the [files](https://drive.google.com/drive/folders/1QmqWfF_dzyZPDIuvkiLHp0X6JiUNbIZI?usp=sharing) and copy them to the `./pretrained_imagenet` folder.\n```shell\nexport CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7\npython DeepAA_search.py --dataset imagenet --n_classes 1000 --use_model resnet50 --n_policies 6 --search_bno 1024 --seed 1 --batch_size 128 --test_batch_size 512 --policy_lr 0.025 --l_mags 13 --use_pool --EXP_G 16 --EXP_gT_factor=4 --train_same_labels 16\n```\n\n### Evaluate the policy found on CIFAR-10/100 and ImageNet. \n```shell\nmkdir ckpt\npython -m DeepAA_evaluate.train -c confs/wresnet28x10_cifar10_DeepAA_1.yaml --dataroot ./data --save ckpt/DeepAA_cifar10.pth --tag Exp_DeepAA_cifar10\npython -m DeepAA_evaluate.train -c confs/wresnet28x10_cifar100_DeepAA_1.yaml --dataroot ./data --save ckpt/DeepAA_cifar100.pth --tag Exp_DeepAA_cifar100\npython -m DeepAA_evaluate.train -c confs/resnet50_imagenet_DeepAA_8x256_1.yaml --dataroot ./data --save ckpt/DeepAA_imagenet.pth --tag Exp_DeepAA_imagenet\n```\n\n### Evaluate the policy found on CIFAR-10/100 with Batch Augmentation.\n```shell\nmkdir ckpt\npython -m DeepAA_evaluate.train -c confs/wresnet28x10_cifar10_DeepAA_BatchAug8x_1.yaml --dataroot ./data --save ckpt/DeepAA_cifar10.pth --tag Exp_DeepAA_cifar10\npython -m DeepAA_evaluate.train -c confs/wresnet28x10_cifar100_DeepAA_BatchAug8x_1.yaml --dataroot ./data --save ckpt/DeepAA_cifar100.pth --tag Exp_DeepAA_cifar100\n```\n\n## Visualization\n\nThe policies found on CIFAR-10/100 and ImageNet are visualized as follows.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/operation_distribution.png\" alt=\"operator\" width=\"70%\"\u003e \u003cbr /\u003e\n\u003c/p\u003e\n\nThe distribution of operations at each layer of the policy for (a) CIFAR-10/100 and (b) ImageNet. The probability of each operation is summed up over all 12 discrete intensity levels of the corresponding transformation.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/magnitude_distribution_cifar.png\" alt=\"magnitude CIFAR\" width=\"70%\"\u003e \u003cbr /\u003e\n\u003c/p\u003e\n\nThe distribution of discrete magnitudes of each augmentation transformation in each layer of the policy for CIFAR-10/100. The x-axis represents the discrete magnitudes and the y-axis represents the probability. The magnitude is discretized to 12 levels with each transformation having its own range. A large absolute value of the magnitude corresponds to high transformation intensity. Note that we do not show identity, autoContrast, invert, equalize, flips, Cutout and crop because they do not have intensity parameters.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/magnitude_distribution_imagenet.png\" alt=\"magnitude ImageNet\" width=\"70%\"\u003e \u003cbr /\u003e\n\u003c/p\u003e\n\nThe distribution of discrete magnitudes of each augmentation transformation in each layer of the policy for ImageNet. The x-axis represents the discrete magnitudes and the y-axis represents the probability. The magnitude is discretized to 12 levels with each transformation having its own range. A large absolute value of the magnitude corresponds to high transformation intensity. Note that we do not show identity, autoContrast, invert, equalize, flips, Cutout and crop because they do not have intensity parameters.\n\n## Citation\nIf you find this useful for your work, please consider citing:\n```\n@inproceedings{\nzheng2022deep,\ntitle={Deep AutoAugment},\nauthor={Yu Zheng and Zhi Zhang and Shen Yan and Mi Zhang},\nbooktitle={International Conference on Learning Representations},\nyear={2022},\nurl={https://openreview.net/forum?id=St-53J9ZARf}\n}\n```\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAIoT-MLSys-Lab%2FDeepAA","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAIoT-MLSys-Lab%2FDeepAA","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAIoT-MLSys-Lab%2FDeepAA/lists"}