{"id":18645052,"url":"https://github.com/fahadshamshad/clip2protect","last_synced_at":"2025-07-27T23:39:47.329Z","repository":{"id":167026772,"uuid":"617857834","full_name":"fahadshamshad/Clip2Protect","owner":"fahadshamshad","description":"[CVPR 2023] Official repository of paper titled \"CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via Adversarial Latent Search\". ","archived":false,"fork":false,"pushed_at":"2024-03-25T00:34:48.000Z","size":46922,"stargazers_count":102,"open_issues_count":4,"forks_count":12,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-25T13:46:21.509Z","etag":null,"topics":["dodging","face-manipulation","face-recognition","impersonation","makeup-transfer","privacy-protection","stylegan","text-guidance","text-guided-image-manipulation","vision-language"],"latest_commit_sha":null,"homepage":"https://fahadshamshad.github.io/Clip2Protect/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fahadshamshad.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-03-23T09:02:35.000Z","updated_at":"2025-01-20T09:11:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"18e09c8d-cb08-4fdb-b2eb-b7ac89937c90","html_url":"https://github.com/fahadshamshad/Clip2Protect","commit_stats":null,"previous_names":["fahadshamshad/clip2protect"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fahadshamshad%2FClip2Protect","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fahadshamshad%2FClip2Protect/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fahadshamshad%2FClip2Protect/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fahadshamshad%2FClip2Protect/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fahadshamshad","download_url":"https://codeload.github.com/fahadshamshad/Clip2Protect/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248401948,"owners_count":21097328,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dodging","face-manipulation","face-recognition","impersonation","makeup-transfer","privacy-protection","stylegan","text-guidance","text-guided-image-manipulation","vision-language"],"created_at":"2024-11-07T06:14:26.219Z","updated_at":"2025-04-11T12:31:13.118Z","avatar_url":"https://github.com/fahadshamshad.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Table of Contents\n\n- [Updates](#updates-loudspeaker)\n- [Central Idea](#central-idea)\n- [Motivation](#motivation-muscle-fire)\n- [Limitation of existing works](#limitation-of-existing-works-warning)\n- [Pipeline](#pipeline)\n- [Instructions for Code usage](#instructions-for-code-usage)\n- [Citation](#citation)\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"imgs/github_logo.jpg\" align=\"center\" width=\"60%\"\u003e\n\n  \u003ch2 align=\"center\"\u003e\u003cstrong\u003eCLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via Adversarial Latent Search [CVPR 2023]\u003c/strong\u003e\u003c/h2\u003e\n\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://fahadshamshad.github.io\"\u003e\u003cstrong\u003e Fahad Shamshad\u003c/strong\u003e\u003c/a\u003e,\n    \u003ca href=\"https://muzammal-naseer.netlify.app/\"\u003e\u003cstrong\u003e Muzammal Naseer\u003c/strong\u003e\u003c/a\u003e,\n    \u003ca href=\"https://scholar.google.com/citations?user=2qx0RnEAAAAJ\u0026hl=en\"\u003e\u003cstrong\u003e Karthik Nandakumar\u003c/strong\u003e\u003c/a\u003e\n    \u003cbr\u003e\n    \u003cspan style=\"font-size:4em; \"\u003e\u003cstrong\u003e MBZUAI, UAE\u003c/strong\u003e.\u003c/span\u003e\n  \u003c/p\u003e\n\u003c/p\u003e\n\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://openaccess.thecvf.com/content/CVPR2023/html/Shamshad_CLIP2Protect_Protecting_Facial_Privacy_Using_Text-Guided_Makeup_via_Adversarial_Latent_CVPR_2023_paper.html\" target='_blank'\u003e\n    \u003cimg src=\"https://img.shields.io/badge/CVPR-Paper-blue.svg\"\u003e\n  \u003c/a\u003e \n  \n  \u003ca href=\"https://fahadshamshad.github.io/Clip2Protect/\" target='_blank'\u003e\n    \u003cimg src=https://img.shields.io/badge/Project-Website-87CEEB\"\u003e\n  \u003c/a\u003e\n\n  \u003ca href=\"https://www.youtube.com/watch?v=CUSVyvM_-6o\" target='_blank'\u003e\n    \u003cimg src=\"https://badges.aleen42.com/src/youtube.svg\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"imgs/github_gifcode.gif\" align=\"center\" width=\"100%\"\u003e\n\u003c/p\u003e\n  \u003cimg src=\"imgs/extra.gif\" align=\"center\" width=\"100%\"\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"imgs/github_anim.gif\" align=\"center\" width=\"60%\"\u003e\n\n\n##  Updates :loudspeaker:\n- **July-19** : Code released.\n- **June-19** : Code and demo release coming soon. Stay tuned!\n\n\n\u003ca name=\"central-idea\"\u003e\u003c/a\u003e\n## 🎯 Central Idea 🎯 \n### We all love sharing photos online, but do you know big companies and even governments can use sneaky 🕵️‍♂️ face recognition software to track us? Our research takes this challenge head-on with a simple and creative idea 🌟: using carefully crafted makeup 💄 to outsmart the tracking software. The cherry on top? We're using everyday, easy-to-understand language 🗣️ to guide the makeup application, giving users much more flexibility! Our approach keeps your photos safe 🛡️ from unwanted trackers without making you look weird or having bizarre patches on your face, issues commonly seen with previous solutions.\n\n\u003ca name=\"motivation-muscle-fire\"\u003e\u003c/a\u003e\n## Motivation :muscle: :fire: \n- Malicious black-box Face recognition systems pose a serious threat to personal security/privacy of **5 billions people** using social media.\n- Unauthorized entities can use FR systems to **track user activities** by scraping face images from social media platforms.\n- There is an urgent demand for effective privacy preservation methods.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"imgs/motivationslides.jpg\" align=\"center\" width=\"100%\"\u003e\n\n## Limitation of existing works :warning: \n - Recent noise-based facial privacy protection approaches result in artefacts. \n - Patch-based privacy approaches provide low privacy protection and their large visible pattern compromises naturalness.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"imgs/limitations.jpg\" align=\"center\" width=\"85%\"\u003e\n\n\n\n\n## Pipeline\n\n**CLIP2Protect** generates face images that look natural and real. **But here's the special part**: it also ensures a high level of privacy protection. This means you can keep sharing images without worrying about unwanted tracking. It consists of two stages. \n- **The latent code initialization stage** reconstructs the given face image in the latent space by fine-tuning the generative model.\n- **The text-guided adversarial optimization stage** utilizes user-defined makeup text prompts and identity-preserving regularization to guide the search for adversarial codes within the latent space to effectively protect the facial privacy.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/static/images/pipeline_color.jpg\" align=\"center\" width=\"95%\"\u003e\n\u003c/p\u003e\n\n\u003ca name=\"instructions-for-code-usage\"\u003e\u003c/a\u003e\n## Intructions for Code usage\n\n### Setup\n\n- **Get code**\n```shell \ngit clone https://github.com/fahadshamshad/Clip2Protect.git\n```\n\n- **Build environment**\n```shell\ncd Clip2Protect\n# use anaconda to build environment \nconda create -n clip2protect python=3.8\nconda activate clip2protect\n# install packages\npip install -r requirements.txt\n```\n\n## Steps for Protecting Faces\n\n1. Our solution relies on the [Rosinality](https://github.com/rosinality/stylegan2-pytorch/) PyTorch implementation of StyleGAN2.\n\n2. **Download the pre-trained StyleGAN2 weights**: \n   - Download the pre-trained StyleGAN2 weights from [here](https://drive.google.com/file/d/1EM87UquaoQmk17Q8d5kYIAHqu0dkYqdT/view?usp=sharing).\n   - Place the weights in the 'pretrained_models' folder.\n\n3. **Download pretrained face recognition models and dataset instructions**:\n   - To acquire pretrained face recognition models and dataset instructions, including target images, please refer to the AMT-GAN page [here](https://github.com/CGCL-codes/AMT-GAN).\n   - Place the pretrained face recognition model in the `models` folder.\n\n4. **Acquire latent codes**:\n   - We assume the latent codes are available in the `latents.pt` file.\n   - You can acquire the latent codes of the face images to be protected using the encoder4editing (e4e) method available [here](https://github.com/omertov/encoder4editing).\n\n5. **Run the code**:\n   - The core functionality is in `main.py`.\n   - Provide the `latents.pt` file and the corresponding faces directory, named 'input_images'.\n   - Generate the protected faces in the 'results' folder by running the following command:\n     ```shell\n     python main.py --data_dir input_images --latent_path latents.pt --protected_face_dir results\n     ```\n\n6. **Generator finetuning and adversarial optimization stages**:\n   - The generator finetuning is implemented in `pivot_tuning.py`.\n   - The adversarial optimization is implemented in `adversarial_optimization.py`.\n\n\n\n\n\n## Citation \n\nIf you're using CLIP2Protect in your research or applications, please cite using this BibTeX:\n```bibtex\n@inproceedings{shamshad2023clip2protect,\n  title={CLIP2Protect: Protecting Facial Privacy Using Text-Guided Makeup via Adversarial Latent Search},\n  author={Shamshad, Fahad and Naseer, Muzammal and Nandakumar, Karthik},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={20595--20605},\n  year={2023}\n  }\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffahadshamshad%2Fclip2protect","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffahadshamshad%2Fclip2protect","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffahadshamshad%2Fclip2protect/lists"}