{"id":13708488,"url":"https://github.com/chandar-lab/PatchUp","last_synced_at":"2025-05-06T13:30:49.040Z","repository":{"id":37647744,"uuid":"271663254","full_name":"chandar-lab/PatchUp","owner":"chandar-lab","description":null,"archived":false,"fork":false,"pushed_at":"2022-12-08T04:21:18.000Z","size":56,"stargazers_count":38,"open_issues_count":5,"forks_count":8,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-08-03T23:12:49.409Z","etag":null,"topics":["convolutional-neural-network","pytorch","regularization-methods"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chandar-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-06-11T23:06:57.000Z","updated_at":"2024-05-16T14:05:56.000Z","dependencies_parsed_at":"2023-01-24T15:45:10.411Z","dependency_job_id":null,"html_url":"https://github.com/chandar-lab/PatchUp","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chandar-lab%2FPatchUp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chandar-lab%2FPatchUp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chandar-lab%2FPatchUp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chandar-lab%2FPatchUp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chandar-lab","download_url":"https://codeload.github.com/chandar-lab/PatchUp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224504122,"owners_count":17322339,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-neural-network","pytorch","regularization-methods"],"created_at":"2024-08-02T23:00:24.032Z","updated_at":"2024-11-13T18:31:24.894Z","avatar_url":"https://github.com/chandar-lab.png","language":"Python","readme":"# PatchUp\n\n## A Regularization Technique for Convolutional Neural Networks\n\nIn this work, we propose PatchUp which is a regularization technique that operates in the hidden space by masking out contiguous blocks of the feature map of a random pair of samples and either mixes (Soft PatchUp) or swaps (Hard PatchUp) these selected contiguous blocks.\nOur experiments verify that Hard PatchUp achieves a better generalization performance in comparison to other state-of-the-art regularization techniques for CNNs like Mixup, cutout, CutMix and ManifoldMixup on CIFAR-10, CIFAR-100, and SVHN datasets. Soft PatchUp provides the second-best performance on CIFAR-10, CIFAR-100 with PreactResnet18, PreactResnet34, and WideResnet-28-10 models and comparable result in SVHN with PreactResnet18, PreactResnet34 with ManifoldMixup. PatchUp provides significant improvements in the generalization on deformed images and better robustness against FGSM adversarial attack. \n\nYo can find further detail on PatchUp in [https://arxiv.org/abs/2006.07794](https://arxiv.org/abs/2006.07794). \n\n### PatchUp Process:\nFollowing image briefly describes how PatchUp works. It is the PatchUp process for two hidden representations associated with two samples randomly selected in the minibatch (a, b). X\u003csub\u003e1\u003c/sub\u003e = g\u003csub\u003ek\u003c/sub\u003e\u003csup\u003e(i)\u003c/sup\u003e(a) and X\u003csub\u003e2\u003c/sub\u003e = g\u003csub\u003ek\u003c/sub\u003e\u003csup\u003e(i)\u003c/sup\u003e(b) where \"i\" is the feature map index. Right top shows Hard PatchUp output and the right bottom shows the interpolated samples with Soft PatchUp. The yellow continuous blocks represent the interpolated selected blocks. \n\n(Figure 1. from the [PatchUp Paper](https://arxiv.org/abs/2006.07794).)\n\u003cbr/\u003e\n\n![patchup_process](https://user-images.githubusercontent.com/38594307/83678128-3b3c0000-a5ab-11ea-8f38-d919ecce8d29.png)\n\n\u003cbr/\u003e\n\n### Citation:\n\nIf you find this work useful and use it in your own research, please consider citing our [paper](https://arxiv.org/abs/2006.07794).\n```\n@misc{faramarzi2020patchup,\n    title={PatchUp: A Regularization Technique for Convolutional Neural Networks},\n    author={Mojtaba Faramarzi and Mohammad Amini and Akilesh Badrinaaraayanan and Vikas Verma and Sarath Chandar},\n    year={2020},\n    eprint={2006.07794},\n    archivePrefix={arXiv},\n    primaryClass={cs.LG}\n}\n```\n\u003cbr/\u003e\n     \n## Project Structure:\nFollowing shows the project structure and modules and files that we have in this project:\n\n\u003cimg width=\"900\" alt=\"project structure\" src=\"https://user-images.githubusercontent.com/38594307/83950930-37eb8300-a7fc-11ea-800a-ebc03bcf41d6.jpg\"\u003e\n\n### Requirements:\nThis code has been tested with python 3.6.9. And, in this implementation some packages were used that you can find names and their version in the requirements.txt.\nTo install the requirements you can do:\n\n```\npip install -r requirements.txt\n```\n\n#### Or install the following packages manually:\n\ntorch==1.3.1\n\u003cbr/\u003e\ntorchvision==0.4.2\n\n#### Additional Packages:\n\nscikit-learn==0.21.3\n\u003cbr/\u003e\nscipy==1.3.1\n\u003cbr/\u003e\nseaborn==0.9.0\n\u003cbr/\u003e\npandas==0.25.3\n\u003cbr/\u003e\npsutil==5.6.7\n\u003cbr/\u003e\nPillow==6.2.1\n\u003cbr/\u003e\npathtools==0.1.2\n\u003cbr/\u003e\nnumpy==1.17.4\n\u003cbr/\u003e\nmatplotlib==3.1.1\n\u003cbr/\u003e\nsix==1.13.0\n\n\n## How to run experiments for CIFAR\n\nFollowing are the experiment commands for CIFAR-10 for the PatchUp, ManifoldMixup, CutMix, cutout, Mixup, and DropBlock.\n\nWe first show how you can run them for CIFAR-10. And then, we indicate the parameter changes that allows you to run experiments on CIFAR-100 and SVHN.\n\n## PatchUp \n\n### CIFAR-10\n\n#### Soft PatchUp:\n\nFor PreActResent models you can run the following command.\n\nNote: X is either preactresnet18 or preactresnet34 and at the end JobID is your job id. \n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/patchup/soft/ --labels_per_class 5000 --valid_labels_per_class 500 --arch \u003cX\u003e  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 2000 --schedule 500 1000 1500 --step_factors 0.1 0.1 0.1 --train patchup --alpha 2.0 --patchup_type soft --patchup_block 7 --patchup_prob 1.0 --gamma 0.75 --job_id \u003cJobID\u003e\n```\nFor WideResNet-28-10:\n\n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/patchup/soft/ --labels_per_class 5000 --valid_labels_per_class 500 --arch wrn28_10  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 400 --schedule 200 300 --step_factors 0.1 0.1 --train patchup --alpha 2.0 --patchup_type soft --patchup_block 7 --patchup_prob 1.0 --gamma 0.75 --job_id \u003cJobID\u003e\n```\n\n#### Hard PatchUp:\n\nFor PreActResent models you can run the following command.\n\nNote: X is either preactresnet18 or preactresnet34 and at the end JobID is your job id. \n\n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/patchup/hard/ --labels_per_class 5000 --valid_labels_per_class 500 --arch \u003cX\u003e  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 2000 --schedule 500 1000 1500 --step_factors 0.1 0.1 0.1 --train patchup --alpha 2.0 --patchup_type hard --patchup_block 7 --patchup_prob 0.7 --gamma 0.5 --job_id \u003cJobID\u003e\n```\n\nFor WideResNet-28-10:\n\n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/patchup/soft/ --labels_per_class 5000 --valid_labels_per_class 500 --arch wrn28_10  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 400 --schedule 200 300 --step_factors 0.1 0.1 --train patchup --alpha 2.0 --patchup_type hard --patchup_block 7 --patchup_prob 0.7 --gamma 0.5 --job_id \u003cJobID\u003e\n```\n\u003cbr/\u003e\n\u003chr/\u003e\n\u003cbr/\u003e\n\n## ManifoldMixup \n\n### CIFAR-10\n\nFor PreActResent models you can run the following command.\n\nNote: X is either preactresnet18 or preactresnet34 and at the end JobID is your job id. \n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/manifold/ --labels_per_class 5000 --valid_labels_per_class 500 --arch \u003cX\u003e  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 2000 --schedule 500 1000 1500 --step_factors 0.1 0.1 0.1 --train manifold_mixup --alpha 1.5 --job_id \u003cJobID\u003e\n```\nFor WideResNet-28-10:\n\n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/patchup/soft/ --labels_per_class 5000 --valid_labels_per_class 500 --arch wrn28_10  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 400 --schedule 200 300 --step_factors 0.1 0.1 --train manifold_mixup --alpha 1.5 --job_id \u003cJobID\u003e\n```\n\u003cbr/\u003e\n\u003chr/\u003e\n\u003cbr/\u003e\n\n## CutMix \n\n### CIFAR-10\n\nFor PreActResent models you can run the following command.\n\nNote: X is either preactresnet18 or preactresnet34 and at the end JobID is your job id. \n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/manifold/ --labels_per_class 5000 --valid_labels_per_class 500 --arch \u003cX\u003e  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 2000 --schedule 500 1000 1500 --step_factors 0.1 0.1 0.1 --train cutmix --cutmix_prob 0.4 --job_id \u003cJobID\u003e\n```\nFor WideResNet-28-10:\n\n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/patchup/soft/ --labels_per_class 5000 --valid_labels_per_class 500 --arch wrn28_10  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 400 --schedule 200 300 --step_factors 0.1 0.1 --train cutmix --cutmix_prob 0.4 --job_id \u003cJobID\u003e\n```\n\u003cbr/\u003e\n\u003chr/\u003e\n\u003cbr/\u003e\n\n## cutout \n\n### CIFAR-10\n\nFor PreActResent models you can run the following command.\n\nNote: X is either preactresnet18 or preactresnet34 and at the end JobID is your job id. \n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/manifold/ --labels_per_class 5000 --valid_labels_per_class 500 --arch \u003cX\u003e  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 2000 --schedule 500 1000 1500 --step_factors 0.1 0.1 0.1 --train cutout --cutout 16 --job_id \u003cJobID\u003e\n```\nFor WideResNet-28-10:\n\n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/patchup/soft/ --labels_per_class 5000 --valid_labels_per_class 500 --arch wrn28_10  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 400 --schedule 200 300 --step_factors 0.1 0.1 --train cutout --cutout 8 --job_id \u003cJobID\u003e\n```\n\nNote: For running the cutout experiment on SVHN, you should set --cutout 20\n\u003cbr/\u003e\n\u003chr/\u003e\n\u003cbr/\u003e\n\n## Mixup \n\n### CIFAR-10\n\nFor PreActResent models you can run the following command.\n\nNote: X is either preactresnet18 or preactresnet34 and at the end JobID is your job id. \n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/manifold/ --labels_per_class 5000 --valid_labels_per_class 500 --arch \u003cX\u003e  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 2000 --schedule 500 1000 1500 --step_factors 0.1 0.1 0.1 --train mixup --alpha 1.0 --job_id \u003cJobID\u003e\n```\nFor WideResNet-28-10:\n\n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/patchup/soft/ --labels_per_class 5000 --valid_labels_per_class 500 --arch wrn28_10  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 400 --schedule 200 300 --step_factors 0.1 0.1 --train mixup --alpha 1.0 --job_id \u003cJobID\u003e\n```\n\u003cbr/\u003e\n\u003chr/\u003e\n\u003cbr/\u003e\n\n## DropBlock \n\n### CIFAR-10\n\nFor PreActResent models you can run the following command.\n\nNote: X is either preactresnet18 or preactresnet34 and at the end JobID is your job id. \n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/manifold/ --labels_per_class 5000 --valid_labels_per_class 500 --arch \u003cX\u003e  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 2000 --schedule 500 1000 1500 --step_factors 0.1 0.1 0.1 --train dropblock --drop_block 7 --keep_prob 0.9 --drop_block_all True --job_id \u003cJobID\u003e\n```\nFor WideResNet-28-10:\n\n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/patchup/soft/ --labels_per_class 5000 --valid_labels_per_class 500 --arch wrn28_10  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 400 --schedule 200 300 --step_factors 0.1 0.1 --train dropblock --drop_block 7 --keep_prob 0.9 --drop_block_all True --job_id \u003cJobID\u003e\n```\n\u003cbr/\u003e\n\u003chr/\u003e\n\u003cbr/\u003e\n\n## Vanilla Model (Model without regularization) \n\n### CIFAR-10\n\nFor PreActResent models you can run the following command.\n\nNote: X is either preactresnet18 or preactresnet34 and at the end JobID is your job id. \n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/manifold/ --labels_per_class 5000 --valid_labels_per_class 500 --arch \u003cX\u003e  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 2000 --schedule 500 1000 1500 --step_factors 0.1 0.1 0.1 --train vanilla --job_id \u003cJobID\u003e\n```\nFor WideResNet-28-10:\n\n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --root_dir ./experiments/patchup/soft/ --labels_per_class 5000 --valid_labels_per_class 500 --arch wrn28_10  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 400 --schedule 200 300 --step_factors 0.1 0.1 --train vanilla --job_id \u003cJobID\u003e\n```\n\u003cbr/\u003e\n\u003chr/\u003e\n\u003cbr/\u003e\n\n## CIFAR-100\n\nFor running experiment on CiFAR-100, you can use above command. But you should change only following parameters:\n```\n* --dataset cifar100 \n* --data_dir ./data/cifar100/\n* --labels_per_class 500 \n* --valid_labels_per_class 50\n```\n\u003cbr/\u003e\n\u003chr/\u003e\n\u003cbr/\u003e\n\n### SVHN\n\nFor running experiment you can use above command. But you should change only following parameters:\n```\n* --dataset svhn \n* --data_dir ./data/svhn/\n* --labels_per_class 7325 \n* --valid_labels_per_class 733\n\nNote: To run the cutout experiment on SVHN, you should  also set --cutout 20\n```\n\u003cbr/\u003e\n\u003chr/\u003e\n\u003cbr/\u003e\n\n# Experiments on Deformed Images\n\nFirst, we need to create an affine transformed test set by running the following command:\n```\npython load_data.py --affine_path ./data/test/affine/\n```\nWe create affine transformed test set described in the paper for the CIFAR-100.\nAfter creating the the Deformed Images test set, \nwe can run generalization experiment on Deformed Images (affine transformed test set) with same commands to train model with a regularization technique with two more parameters. Following is one command example that is used in Soft PatchUp.\n```\npython main.py --dataset cifar100 --data_dir ./data/cifar100/ --affine_test --affine_path ./data/test/affine/  --root_dir ./experiments/patchup/soft/ --labels_per_class 500 --valid_labels_per_class 50 --arch wrn28_10  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 400 --schedule 200 300 --step_factors 0.1 0.1 --train patchup --alpha 2.0 --patchup_type soft --patchup_block 7 --patchup_prob 1.0 --gamma 0.25 --job_id \u003cJobID\u003e\n```\nNote: Use the above as a pattern to create a command to run an experiment for evaluating the performance of other approaches in this task.\n\u003cbr/\u003e\n\u003chr/\u003e\n\u003cbr/\u003e\n\n# Robustness to Adversarial Examples\n\nin order to see the regularized models' robustness against the FGSM attack, we can use following parameter:\n* --fsgm_attack True\n\nThe following command runs this experiment on PreActResNet18 in CIFAR-10 with Soft PatchUp and evaluate its robustness against the FGSM attack.\n```\npython main.py --dataset cifar10 --data_dir ./data/cifar10/ --fsgm_attack True --root_dir ./experiments/patchup/soft/ --labels_per_class 5000 --valid_labels_per_class 500 --arch \u003cX\u003e  --learning_rate 0.1 --momentum 0.9 --decay 0.0001 --epochs 2000 --schedule 500 1000 1500 --step_factors 0.1 0.1 0.1 --train patchup --alpha 2.0 --patchup_type soft --patchup_block 7 --patchup_prob 1.0 --gamma 0.25 --job_id \u003cJobID\u003e\n```\nNote: Use above as an example command for running experiment on evaluating other approaches performance.\n\n\n\n\n","funding_links":[],"categories":["Table of Contents"],"sub_categories":["Sample Mixup Policies in SL"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchandar-lab%2FPatchUp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchandar-lab%2FPatchUp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchandar-lab%2FPatchUp/lists"}