{"id":13935585,"url":"https://github.com/ai-forever/ghost","last_synced_at":"2025-05-14T15:06:11.003Z","repository":{"id":38413399,"uuid":"437939371","full_name":"ai-forever/ghost","owner":"ai-forever","description":"A new one shot face swap approach for image and video domains","archived":false,"fork":false,"pushed_at":"2025-02-25T10:15:05.000Z","size":96160,"stargazers_count":1422,"open_issues_count":69,"forks_count":281,"subscribers_count":26,"default_branch":"main","last_synced_at":"2025-05-14T02:47:53.823Z","etag":null,"topics":["computer-vision","deep-face-swap","deep-learning","deepfake","face-swap","faceswap","ghost","ghost-faceswap","ghost-swap","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ai-forever.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-13T16:09:59.000Z","updated_at":"2025-05-12T13:56:11.000Z","dependencies_parsed_at":"2025-03-10T21:46:55.459Z","dependency_job_id":"fa0fc68c-8855-4a0e-aa59-013708194308","html_url":"https://github.com/ai-forever/ghost","commit_stats":{"total_commits":34,"total_committers":4,"mean_commits":8.5,"dds":0.5588235294117647,"last_synced_commit":"ea23cce2543a5933a22cbe98bb2973b8550a3ba1"},"previous_names":["ai-forever/sber-swap"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fghost","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fghost/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fghost/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fghost/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ai-forever","download_url":"https://codeload.github.com/ai-forever/ghost/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254169035,"owners_count":22026207,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","deep-face-swap","deep-learning","deepfake","face-swap","faceswap","ghost","ghost-faceswap","ghost-swap","pytorch"],"created_at":"2024-08-07T23:01:54.472Z","updated_at":"2025-05-14T15:06:10.947Z","avatar_url":"https://github.com/ai-forever.png","language":"Python","readme":"[[Paper](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=\u0026arnumber=9851423)] [[Habr](https://habr.com/ru/company/sberbank/blog/645919/)]\n\n# 👻 GHOST: Generative High-fidelity One Shot Transfer \n\nOur paper [\"GHOST—A New Face Swap Approach for Image and Video Domains\"](https://ieeexplore.ieee.org/abstract/document/9851423) has been published on IEEE Xplore.\n\n\u003cp align=\"left\"\u003e\n  Google Colab Demo\n\u003c/p\u003e\n\u003cp align=\"left\"\u003e\n  \u003ca href=\"https://colab.research.google.com/drive/1vXTpsENipTmjTMggwveCkXASwxUk270n\"\u003e\n  \u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\"/\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n## GHOST Ethics \n\nDeepfake stands for a face swapping algorithm where the source and target can be an image or a video. Researchers have investigated sophisticated generative adversarial networks (GAN), autoencoders, and other approaches to establish precise and robust algorithms for face swapping. However, the achieved results are far from perfect in terms of human and visual evaluation. In this study, we propose a new one-shot pipeline for image-to-image and image-to-video face swap solutions - GHOST (Generative High-fidelity One Shot Transfer).\n\nDeep fake synthesis methods have been improved a lot in quality in recent years. The research solutions were wrapped in easy-to-use API, software and different plugins for people with a little technical knowledge. As a result, almost anyone is able to make a deepfake image or video by just doing a short list of simple operations. At the same time, a lot of people with malicious intent are able to use this technology in order to produce harmful content. High distribution of such a content over the web leads to caution, disfavor and other negative feedback to deepfake synthesis or face swap research.\n\nAs a group of researchers, we are not trying to denigrate celebrities and statesmen or to demean anyone. We are computer vision researchers, we are engineers, we are activists, we are hobbyists, we are human beings. To this end, we feel that it's time to come out with a standard statement of what this technology is and isn't as far as us researchers are concerned.\n* GHOST is not for creating inappropriate content.\n* GHOST is not for changing faces without consent or with the intent of hiding its use.\n* GHOST is not for any illicit, unethical, or questionable purposes.\n* GHOST exists to experiment and discover AI techniques, for social or political commentary, for movies, and for any number of ethical and reasonable uses.\n\nWe are very troubled by the fact that GHOST can be used for unethical and disreputable things. However, we support the development of tools and techniques that can be used ethically as well as provide education and experience in AI for anyone who wants to learn it hands-on. Now and further, we take a **zero-tolerance approach** and **total disregard** to anyone using this software for any unethical purposes and will actively discourage any such uses.\n\n\n## Image Swap Results \n![](/examples/images/example1.png)\n\n![](/examples/images/example2.png)\n\n## Video Swap Results\n\u003cdiv\u003e\n\u003cimg src=\"/examples/videos/orig.webp\" width=\"360\"/\u003e\n\u003cimg src=\"/examples/videos/elon.webp\" width=\"360\"/\u003e\n\u003cimg src=\"/examples/videos/khabenskii.webp\" width=\"360\"/\u003e\n\u003cimg src=\"/examples/videos/mark.webp\" width=\"360\"/\u003e\n\u003cdiv/\u003e\n\n## Installation\n  \n1. Clone this repository\n  ```bash\n  git clone https://github.com/sberbank-ai/sber-swap.git\n  cd sber-swap\n  git submodule init\n  git submodule update\n  ```\n2. Install dependent packages\n  ```bash\n  pip install -r requirements.txt\n  ```\n  If it is not possible to install onnxruntime-gpu, try onnxruntime instead  \n  \n3. Download weights\n  ```bash\n  sh download_models.sh\n  ```\n## Usage\n  1. Colab Demo \u003ca href=\"https://colab.research.google.com/drive/1B-2JoRxZZwrY2eK_E7TB5VYcae3EjQ1f\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"google colab logo\"\u003e\u003c/a\u003e or you can use jupyter notebook [SberSwapInference.ipynb](SberSwapInference.ipynb) locally\n  2. Face Swap On Video\n  \n  Swap to one specific person in the video. You must set face from the target video (for example, crop from any frame).\n  ```bash\n  python inference.py --source_paths {PATH_TO_IMAGE} --target_faces_paths {PATH_TO_IMAGE} --target_video {PATH_TO_VIDEO}\n  ```\n  Swap to many person in the video. You must set multiple faces for source and the corresponding multiple faces from the target video.\n  ```bash\n  python inference.py --source_paths {PATH_TO_IMAGE PATH_TO_IMAGE ...} --target_faces_paths {PATH_TO_IMAGE PATH_TO_IMAGE ...} --target_video {PATH_TO_VIDEO}\n  ```\n  3. Face Swap On Image\n  \n  You may set the target face, and then source will be swapped on this person, or you may skip this parameter, and then source will be swapped on any person in the image.\n  ```bash\n  python inference.py --target_path {PATH_TO_IMAGE} --image_to_image True\n  ```\n  \n## Training\n  \nWe also provide the training code for face swap model as follows:\n  1. Download [VGGFace2 Dataset](https://www.robots.ox.ac.uk/~vgg/data/vgg_face/).\n  2. Crop and align faces with out detection model.\n  ```bash\n  python preprocess_vgg.py --path_to_dataset {PATH_TO_DATASET} --save_path {SAVE_PATH}\n  ```\n  3. Start training. \n  ```bash\n  python train.py --run_name {YOUR_RUN_NAME}\n  ```\nWe provide a lot of different options for the training. More info about each option you can find in `train.py` file. If you would like to use wandb logging of the experiments, you should login to wandb first  `--wandb login`.\n  \n### Tips\n  1. For the first epochs we suggest not to use eye detection loss and scheduler if you train from scratch.\n  2. In case of finetuning you can variate losses coefficients to make the output look similar to the source identity, or vice versa, to save features and attributes of target face.\n  3. You can change the backbone of the attribute encoder and num_blocks of AAD ResBlk using parameters `--backbone` and `--num_blocks`.\n  4. During the finetuning stage you can use our pretrain weights for generator and discriminator that are located in `weights` folder. We provide the weights for models with U-Net backbone and 1-3 blocks in AAD ResBlk. The main model architecture contains 2 blocks in AAD ResBlk.\n  \n## Cite\nIf you use our model in your research, we would appreciate using the following citation\n\n  ### BibTeX Citation\n  ```\n  @article{9851423,  \n           author={Groshev, Alexander and Maltseva, Anastasia and Chesakov, Daniil and Kuznetsov, Andrey and Dimitrov, Denis},  \n           journal={IEEE Access},   \n           title={GHOST—A New Face Swap Approach for Image and Video Domains},   \n           year={2022},  \n           volume={10},  \n           number={},  \n           pages={83452-83462},  \n           doi={10.1109/ACCESS.2022.3196668}\n  }\n  ```\n  \n  ### General Citation\n  \nA. Groshev, A. Maltseva, D. Chesakov, A. Kuznetsov and D. Dimitrov, \"GHOST—A New Face Swap Approach for Image and Video Domains,\" in IEEE Access, vol. 10, pp. 83452-83462, 2022, doi: 10.1109/ACCESS.2022.3196668.\n  \n","funding_links":[],"categories":["Python","Tools \u0026 Frameworks"],"sub_categories":["Deep Fake"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fai-forever%2Fghost","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fai-forever%2Fghost","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fai-forever%2Fghost/lists"}