{"id":19260339,"url":"https://github.com/mitre/inap","last_synced_at":"2025-04-21T16:31:40.952Z","repository":{"id":209503533,"uuid":"721823395","full_name":"mitre/inap","owner":"mitre","description":"A pipeline for generating inconspicuous naturalistic adversarial patches (INAPs) against object detectors with one input image ","archived":false,"fork":false,"pushed_at":"2024-09-09T20:23:38.000Z","size":13193,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-04-01T14:22:16.378Z","etag":null,"topics":["adversarial-attacks","adversarial-machine-learning","adversarial-patches","generative-adversarial-network","object-detection"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mitre.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-21T21:02:03.000Z","updated_at":"2024-09-09T20:23:42.000Z","dependencies_parsed_at":null,"dependency_job_id":"842c586c-a6ef-41b0-abe1-4245c69a42f4","html_url":"https://github.com/mitre/inap","commit_stats":null,"previous_names":["mitre/inap"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitre%2Finap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitre%2Finap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitre%2Finap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitre%2Finap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mitre","download_url":"https://codeload.github.com/mitre/inap/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250090870,"owners_count":21373267,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-attacks","adversarial-machine-learning","adversarial-patches","generative-adversarial-network","object-detection"],"created_at":"2024-11-09T19:20:04.031Z","updated_at":"2025-04-21T16:31:35.941Z","avatar_url":"https://github.com/mitre.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Inconspicuous Naturalistic Adversarial Patches (INAPs) for Fooling Object Detectors\nThis project uses code from [this repository](https://github.com/zalandoresearch/famos.git) by Zolando Research.\n\nThis code base introduces a generative adversarial network (GAN) based approach to generating inconspicuous naturalistic adversarial patches (INAPs), which are intended to degrade the performance of object detection algorithms while remaining difficult to detect visually. This approach is separate from other work on traditional adversarial patches for several reasons. First, this project targets object detection model failure rather than classification. Secondly, this work focuses on adversarial patches that are placed in the background of a scene, rather than on top of the subjects in a scene. Additionally, the GAN architecture utilized for INAP creation in this work only requires a single image input for training.\n\nPlease refer to [\"Learning Texture Manifolds with the Periodic Spatial GAN\"](http://arxiv.org/abs/1811.09236) for an in-depth explanation of periodic spatial GANs and alternate use cases.\n\nThe code base has been tested on Ubuntu 18.04, PyTorch 1.12, and Python 3.9 utilizing NVIDIA Tesla V100-SXM2-32GB GPUs. It is recommended to have a GPU with 24GB or more for successful training.\n\n## Applications\nWith this code, you can train a generative model on a single image to generate an inconspicuous adversarial patch for a selected location on the input image. \n\n### Example 1\n\n| Original Image Detections| Attacked Image Detections |INAP |Result |\n|--|--|--|--|\n| \u003cimg src=\"./content/readme/original_sample1.png\" width=\"250\"\u003e | \u003cimg src=\"./content/readme/attacked_sample1.png\" width=\"250\"\u003e | \u003cimg src=\"./content/readme/inap_sample1.png\" width=\"200\"\u003e | 1 person hallucination of 64% |\n\n\n### Example 2\n\n| Original Image Detections| Attacked Image Detections |INAP |Result |\n|--|--|--|--|\n| \u003cimg src=\"./content/readme/original_sample2.png\" width=\"250\"\u003e | \u003cimg src=\"./content/readme/attacked_sample2.png\" width=\"250\"\u003e | \u003cimg src=\"./content/readme/inap_sample2.png\" width=\"200\"\u003e |3 person hallucinations of 93%, 93%, and 76% |\n\n\n### Example 3\n\n| Original Image Detections| Attacked Image Detections |INAP |Result |\n|--|--|--|--|\n| \u003cimg src=\"./content/readme/original_sample3.png\" width=\"200\"\u003e | \u003cimg src=\"./content/readme/attacked_sample3.png\" width=\"200\"\u003e | \u003cimg src=\"./content/readme/inap_sample3.png\" width=\"200\"\u003e | 1 person hallucination of 47%|\n\n\n## Code Usage\n\n### Install Dependencies\nThe base requirements to run the training script is listed in the requirements.txt file.\n\nTo install the CUDA compatible torch and torchvision modules, you can run the following command: \n`pip3 install torch==1.12.1+cu113 torchvision==0.13.1+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html`\n\n### Select the context\nSelect your context image. Some example images are provided in the `./content/` folder.\n\nSelect where you would like the patch to be on the context images, and record the coordinates of the area in the format: `[left, top, right, bottom]`. Please select patch coordinates that create a square with width and height a multiple of 32 (e.g. 96, 128, 160, ..., 288, 320, etc.).\n\n### Train \nUpdate the config.py file to point the coordinates (`--coords`) and target image (`--contentPath`) arguments to the correct value.\nRun `python train.py` in the command line. You may run the code on your desired GPU with the `--device` flag specified in the command line. \n\nFor example: \n`python train.py --device=2`\n\nOnce you start training, you can track progress in the `results/` folder of the associated process.\n\nDepending on the input size of the patch and image, your GAN may take between a few hours to a whole day to learn and may be too much data for a less powerful GPU. Be mindful of this when choosing your inputs. You can toggle the `--cropContentDisciminator` parameter in the config.py file to speed up training time. This setting crops the image being passed through the GAN to be only the surrounding area of the patch location.\n\n## Parameters\n\nThere are many different parameters that can be tweaked for training the GAN. You can locate and change these in the config.py file. \n\nSome common parameters to experiment with include:\n- Batch Size (`--batchSize`)\n- Learning Rate (`--lr`)\n- Number of Epochs (`--niter`)\n- Alternative Loss Function (`--WGAN`, `--LS`)\n- Adversarial Weight ( `--hp_od` )\n\nAdditionally, you may choose to switch on the printability parameter (`--printability`), which will compare the GAN generated patch pixel values to real world printer colors. You may utilize the provided color_palette.txt for this, or you may use your own.\n\n## Authors and Acknowledgement\nSpecial thanks to Mike Tan (ytan@mitre.org), Omid Aramoon(oaramoon@mitre.org), and Frank Chang (frank.chang1000@gmail.com) for their contributions to this work.\n\n## Notice\nApproved for Public Release; Distribution Unlimited. Public Release Case Number 23-3720\n©2023 The MITRE Corporation. ALL RIGHTS RESERVED.\n\n## License\nThis work is published under the apache-2.0 license. Please see [LICENSE](./LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitre%2Finap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmitre%2Finap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitre%2Finap/lists"}