{"id":19146455,"url":"https://github.com/kahsolt/adv-patch-defense","last_synced_at":"2026-06-12T00:30:30.278Z","repository":{"id":171952126,"uuid":"636266339","full_name":"Kahsolt/adv-patch-defense","owner":"Kahsolt","description":"Adversarial Patch defense using SegmentAndComplete (SAC) \u0026 Masked AutoEncoder (MAE)","archived":false,"fork":false,"pushed_at":"2025-01-25T09:18:49.000Z","size":2991,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-25T10:19:36.605Z","etag":null,"topics":["adversarial-defense","adversarial-patches","mae","sac"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Kahsolt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-04T13:18:07.000Z","updated_at":"2025-01-25T09:18:53.000Z","dependencies_parsed_at":null,"dependency_job_id":"2ff86611-c27c-40d4-bdfd-1a9cc143fffd","html_url":"https://github.com/Kahsolt/adv-patch-defense","commit_stats":null,"previous_names":["kahsolt/adv-patch-defense"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kahsolt%2Fadv-patch-defense","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kahsolt%2Fadv-patch-defense/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kahsolt%2Fadv-patch-defense/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kahsolt%2Fadv-patch-defense/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Kahsolt","download_url":"https://codeload.github.com/Kahsolt/adv-patch-defense/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240229935,"owners_count":19768588,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-defense","adversarial-patches","mae","sac"],"created_at":"2024-11-09T07:44:11.782Z","updated_at":"2026-06-12T00:30:29.346Z","avatar_url":"https://github.com/Kahsolt.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# adv-patch-defense\n\n    Adversarial Patch defense using SegmentAndComplete (SAC) \u0026 Masked AutoEncoder (MAE)\n\n----\n\nNaive defense pipeline to AdvPatch := SAC + MAE\n\n  - Consider image classification and object dectection task\n  - AdvPatch puts high-freq color patches on given image (circle/square, ~15%(?) full area)\n  - SAC detects adv patches then mask with pure black (round-corner square, nearly cover)\n  - MAE recovers the black area\n\nPartial result (`limit=256`):\n\n| Setting | Accuracy | cmdline |\n| :-: | :-: | :-: | :-: |\n| clean   | 96.484% | `python run_adv_patch.py -L 256 -B 32` |\n| attack  | 44.531% | `python run_adv_patch.py -L 256 -B 32 --ap` |\n| attack  | 46.484% | `python run_adv_patch.py -L 256 -B 32 --ap --ap_pgd` |\n| defense |         | `python run_adv_patch.py -L 256 -B 4 --ap --sac --mae` |\n| defense |         | `python run_adv_patch.py -L 256 -B 4 --ap --sac --sac_complete --mae` |\n\n⚠ For 12G VRAM, max `batch_size` for `attack/defense` is `48/6` separately\n⚠ To defend against `--ip` attack, you must turn on `--sac_complete` because the pre-generated patches are not that PGD-like noisy\n\n⚪ run error trouble shoot\n\nQ: pip package version issues  \nA: assure `timm==0.3.2`, `torch\u003e1.7`; recommend `numpy==1.24.3`, `torch==2.0.1+cu117`, `torchvision==0.15.2+cu117`  \n\nQ: ImportError in site-package `timm`  \nA: modify `timm/models/layers/helper.py:6` to `import collections.abc as container_abcs`  \n\nQ: deprecation numpy error in local `mae` repo  \nA: modify `mae/util/pos_embed.py:56`, change `np.float` to `np.float32`  \n\n\n### quick start\n\n- run `repo/init_repos.cmd`\n- download [MAE weights](https://dl.fbaipublicfiles.com/mae/visualize/mae_visualize_vit_large_ganloss.pth) to `repo/mae/models`\n- download test data [ImageNet-1k]()\n- run clean test: `python run_adv_patch.py`\n- run attack test: `python run_adv_patch.py --ap` or `python run_adv_patch.py --ip`\n- run defense test\n  - batch: `python run_adv_patch.py -B 16 -L 16 --ap --sac --mae`\n  - all:   `python run_adv_patch.py --ap --sac --mae`\n\n\npipeline subsitutes:\n\n```\n⚪ adv patch attack\n  - AdvPatch (*)\n  - ImageNet-Patch\n  - DPatch / Robust-DPatch\n  - regioned PGD\n  - feature_level_adv\n\n⚪ adv patch detect (mask / drop)\n  - SAC (*)\n  - smoothed-vit\n  - PatchCleanser\n\n⚪ high-freq suppress\n  - PatchGuard\n  - local gradients smoothing\n  - unsharp mask (new = low + high * alpha)\n\n⚪ image recover\n  - MAE / dMAE (*)\n  - MST / ZITS inpainting\n  - DDNM (null-space diffusion inpainting)\n```\n\n### ImageNet-Patch info\n\n```python\n{\n  'patch_size': 50, \n  'target_names': {\n    804: 'soap dispenser', \n    513: 'cornet', \n    923: 'plate', \n    954: 'banana', \n    968: 'cup', \n    878: 'typewriter keyboard', \n    546: 'electric guitar', \n    585: 'hair spray', \n    806: 'sock', \n    487: 'cellular telephone'}, \n  'patch_type': 'square', \n  'input_shape': (3, 224, 224)\n}\n```\n\n### reference\n\n⚪ adv attack toolbox\n\n- adversarial-robustness-toolbox: [https://github.com/Trusted-AI/adversarial-robustness-toolbox](https://github.com/Trusted-AI/adversarial-robustness-toolbox)\n  - 很全面的攻防框架！！\n- cleverhans: [https://github.com/cleverhans-lab/cleverhans](https://github.com/cleverhans-lab/cleverhans)\n  - 已停止更新，实现的攻击比 torchattacks 还少\n\n⚪ patch attack\n\n- ImageNet-Patch: [https://github.com/pralab/ImageNet-Patch](https://github.com/pralab/ImageNet-Patch)\n  - 基于 ImageNet 预制作的 10 类 patch\n- AdvPattern: [https://github.com/whuAdv/AdvPattern](https://github.com/whuAdv/AdvPattern)\n  - 攻击行人重识别模型，衣服上贴补丁，不知所云的方法\n- feature_level_adv: [https://github.com/thestephencasper/feature_level_adv](https://github.com/thestephencasper/feature_level_adv)\n  - 使用GAN去生成带补丁的对抗样本，欺骗一群分类器；很鸡贼的数据来源\n\n⚪ image recover (patch \u0026 inpaint)\n\n- mae: [https://github.com/facebookresearch/mae](https://github.com/facebookresearch/mae)\n- dmae: [https://github.com/quanlin-wu/dmae](https://github.com/quanlin-wu/dmae)\n  - 数据集上加高斯降噪再训练的MAE, 没用GANLoss，恢复效果很烂\n- MST_inpainting: [https://github.com/ewrfcas/MST_inpainting](https://github.com/ewrfcas/MST_inpainting)\n- ZITS_inpainting: [https://github.com/DQiaole/ZITS_inpainting](https://github.com/DQiaole/ZITS_inpainting)\n  - Transformer 擦除补全模型\n\n⚪ defense\n\n- phattacks: [https://github.com/tongwu2020/phattacks](https://github.com/tongwu2020/phattacks)\n  - 物理防御，使用区块化PGD的对抗训练；水文\n- smoothed-vit: [https://github.com/MadryLab/smoothed-vit](https://github.com/MadryLab/smoothed-vit)\n  - 对抗补丁防御：原图纵向切竖条，对每个竖条过VIT做分类，最后投票聚合\n- SegmentAndComplete: [https://github.com/joellliu/SegmentAndComplete](https://github.com/joellliu/SegmentAndComplete)\n  - 对抗补丁位置检测，移除\n- PatchGuard： [https://github.com/inspire-group/PatchGuard](https://github.com/inspire-group/PatchGuard)\n  - 使用更小的感受野，特征图数值下截断0、大激活值直接改成0\n- PatchCleanser: [https://github.com/inspire-group/PatchCleanser](https://github.com/inspire-group/PatchCleanser)\n  - 用一个比攻击者补丁更大的补丁轮询遮盖原图，若所有输出不完全一致则保持第一个补丁再用第二个补丁轮询；纯理论意淫，sb工作\n- local_gradients_smoothing: [https://github.com/fabiobrau/local_gradients_smoothing](https://github.com/fabiobrau/local_gradients_smoothing)\n  - 传统数值方法(Sobel)计算一张图的局域梯度，修改原图压抑梯度大的部分（线性，梯度越大被压抑到越小）\n  - 开源代码似乎并未实现论文全部演示\n\n----\n\nby Armit\n2023/03/23 \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkahsolt%2Fadv-patch-defense","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkahsolt%2Fadv-patch-defense","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkahsolt%2Fadv-patch-defense/lists"}