{"id":13487819,"url":"https://github.com/LPengYang/FreeDrag","last_synced_at":"2025-03-27T23:31:48.571Z","repository":{"id":178956877,"uuid":"662068815","full_name":"LPengYang/FreeDrag","owner":"LPengYang","description":"Official Implementation of FreeDrag (CVPR 2024)","archived":false,"fork":false,"pushed_at":"2025-01-23T00:56:15.000Z","size":122406,"stargazers_count":413,"open_issues_count":0,"forks_count":20,"subscribers_count":19,"default_branch":"main","last_synced_at":"2025-01-23T01:30:38.920Z","etag":null,"topics":["draggan","image-editing","image-generation","image-manipulation"],"latest_commit_sha":null,"homepage":"https://lin-chen.site/projects/freedrag/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LPengYang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-04T09:36:38.000Z","updated_at":"2025-01-23T00:56:18.000Z","dependencies_parsed_at":null,"dependency_job_id":"70030e54-797a-4e71-8f50-d36665f953e6","html_url":"https://github.com/LPengYang/FreeDrag","commit_stats":null,"previous_names":["lpengyang/freedrag"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LPengYang%2FFreeDrag","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LPengYang%2FFreeDrag/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LPengYang%2FFreeDrag/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LPengYang%2FFreeDrag/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LPengYang","download_url":"https://codeload.github.com/LPengYang/FreeDrag/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245944020,"owners_count":20697945,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["draggan","image-editing","image-generation","image-manipulation"],"created_at":"2024-07-31T18:01:04.525Z","updated_at":"2025-03-27T23:31:43.540Z","avatar_url":"https://github.com/LPengYang.png","language":"Python","funding_links":[],"categories":["Personalized Restoration","Datasets"],"sub_categories":["Region-based Drag"],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./resources/logo2.png\"\u003e\n\u003c/p\u003e\n\n# FreeDrag: Feature Dragging for Reliable Point-based Image Editing  (CVPR 2024)\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./resources/Teaser.png\"\u003e\n  \u003cimg src=\"./resources/comparison_diffusion_2.png\"\u003e\n\u003c/p\u003e\n\n## Visualization \n[![]](https://user-images.githubusercontent.com/58554846/253733958-c97629a0-5928-476b-99f2-79d5f92762e7.mp4)\n\n\n## Web Demo (online dragging editing in 11 different StyleGAN2 models)\n[![Open in OpenXLab](https://cdn-static.openxlab.org.cn/app-center/openxlab_app.svg)](https://openxlab.org.cn/apps/detail/LPengYang/FreeDrag)\n\nOfficial implementation of **FreeDrag: Feature Dragging for Reliable Point-based Image Editing**.\n- *Authors*: Pengyang Ling*, [Lin Chen*](https://lin-chen.site), [Pan Zhang](https://panzhang0212.github.io/), Huaian Chen, Yi Jin, Jinjin Zheng,\n- *Institutes*: University of Science and Technology of China; Shanghai AI Laboratory\n- [[Paper]](https://arxiv.org/abs/2307.04684) [[Project Page]](https://lin-chen.site/projects/freedrag) [[Web Demo]](https://openxlab.org.cn/apps/detail/LPengYang/FreeDrag)\n\nThis repo proposes FreeDrag, a novel interactive point-based image editing framework free of the laborious and unstable point tracking process🔥🔥🔥.\n\n\n## Abstract\nTo serve the intricate and varied demands of image editing, precise and flexible manipulation in image content is indispensable. Recently, Drag-based editing methods have gained impressive performance. However, these methods predominantly center on point dragging, resulting in two noteworthy drawbacks, namely \"miss tracking\", where difficulties arise in accurately tracking the predetermined handle points, and \"ambiguous tracking\", where tracked points are potentially positioned in wrong regions that closely resemble the handle points. To address the above issues, we propose **FreeDrag**, a feature dragging methodology designed to free the burden on point tracking. The **FreeDrag** incorporates two key designs, i.e., template feature via adaptive updating and line search with backtracking, the former improves the stability against drastic content change by elaborately controls feature updating scale after each dragging, while the latter alleviates the misguidance from similar points by actively restricting the search area in a line. These two technologies together contribute to a more stable semantic dragging with higher efficiency. Comprehensive experimental results substantiate that our approach significantly outperforms pre-existing methodologies, offering reliable point-based editing even in various complex scenarios.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./resources/comparison_gan.png\"\u003e\n  \u003cimg src=\"./resources/comparison_diffusion_1.png\"\u003e\n\u003c/p\u003e\n\n## 📜 News\n[2024/03/06] [FreeDrag](https://arxiv.org/abs/2307.04684) is accepted by CVPR 2024.\n\n[2023/12/11] The updataed [FreeDrag](https://arxiv.org/abs/2307.04684) containing the implementations in both StyleGAN and Diffusion models is available now.\n\n[2023/12/8]  FreeDrag based on diffusion model is available now, which support dragging editing in both real images and generated images.\n\n[2023/7/31] The web demo (StyleGAN) in [OpenXLab](https://openxlab.org.cn/apps/detail/LPengYang/FreeDrag) is available now.\n\n[2023/7/28] The function of real image editing is available now.\n\n[2023/7/15] Code of local demo is available now!💥\n\n[2023/7/11] The [paper](https://arxiv.org/abs/2307.04684) and [project page](https://lin-chen.site/projects/freedrag) are released!\n\n## 💡 Highlights\n- [x] Local demo of FreeDrag\n- [x] Web demo of FreeDrag\n- [x] Diffusion-based FreeDrag\n\n## 🛠️Usage\n\nFirst clone our repository\n```\ngit clone --depth=1 https://github.com/LPengYang/FreeDrag\n```\nTo create a new environment, please follow the requirements of [NVlabs/stylegan2-ada](https://github.com/NVlabs/stylegan2-ada-pytorch#requirements). \n\n**Notice:** It is observed that the errors (setting up PyTorch plugin “bias_act_plugin“... Failed or “upfirdn2d_plugin“... Failed) may appear in some devices, we hope these potential solutions ([1](https://blog.csdn.net/qq_15969343/article/details/129190607), [2](https://github.com/NVlabs/stylegan2-ada-pytorch/issues/155), [3](https://github.com/NVlabs/stylegan3/issues/124), [4](https://github.com/XingangPan/DragGAN/issues/106)) could be helpful in this case.\n\nThen install the additional requirements\n\n```\npip install -r requirements.txt\n```\n\nThen download the pre-trained models of stylegan2\n```\nbash download_models.sh\n```\n**Notice：**  The first model (face model) could be downloaded very slowly in some cases. In this case, it is suggested to restart the download (works sometimes) or directly download it from this [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/research/models/stylegan2/files), please download the correct model (ffhq-512×512) and renamed it as \"faces.pkl\" and manually put it in the created checkpoints file (after all the other models are downloaded).\n\nFinally initialize the gradio platform for interactive point-based manipulation\n\n```\nCUDA_LAUNCH_BLOCKING=1 python FreeDrag_gradio.py\n```\nYou can also upload your images and then edit them. For a high-quality image inversion, it is suggested to make sure that the resolution and style (such as layout) of the uploaded images are consistent with the generated images of corresponding model. The resolution of different model is listed as follows:\n\n|Model|face|horse|elephant|lion|dog|bicycle|giraffe|cat|car|church|metface|\n|:----:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|Resolution|512|256|512|512|1024|256|512|512|512|256|1024|\n\nThe proposed **FreeDragBench Dataset** is available on the [website](https://drive.google.com/file/d/1p2muR6aW6fqEGW8yTcHl86DCuUkwgtNY/view?usp=sharing).\n\n## ❤️Acknowledgments\n- [DragGAN](https://github.com/XingangPan/DragGAN/)\n- [DragDiffusion](https://yujun-shi.github.io/projects/dragdiffusion.html)\n- [StyleGAN2](https://github.com/NVlabs/stylegan2-ada-pytorch)\n  \n## License\nAll codes used or modified from [StyleGAN2](https://github.com/NVlabs/stylegan2-ada-pytorch) are under the [Nvidia Source Code License](https://github.com/NVlabs/stylegan3/blob/main/LICENSE.txt). \nThe code related to the FreeDrag algorithm is only allowed for personal activity. The diffusion-based FreeDrag is implemented based on [DragDiffusion](https://yujun-shi.github.io/projects/dragdiffusion.html).\n\n## ✒️ Citation\nIf you find our work helpful for your research, please consider citing the following BibTeX entry.\n```bibtex\n@article{ling2023freedrag,\n  title={Freedrag: Point tracking is not you need for interactive point-based image editing},\n  author={Ling, Pengyang and Chen, Lin and Zhang, Pan and Chen, Huaian and Jin, Yi},\n  journal={arXiv preprint arXiv:2307.04684},\n  year={2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLPengYang%2FFreeDrag","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FLPengYang%2FFreeDrag","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLPengYang%2FFreeDrag/lists"}