{"id":22756958,"url":"https://github.com/chinmaynehate/dfspot-deepfake-recognition","last_synced_at":"2025-10-04T21:05:38.644Z","repository":{"id":41905929,"uuid":"432826636","full_name":"chinmaynehate/DFSpot-Deepfake-Recognition","owner":"chinmaynehate","description":"Determine whether a given video sequence has been manipulated or synthetically generated","archived":false,"fork":false,"pushed_at":"2022-10-21T20:47:55.000Z","size":27155,"stargazers_count":98,"open_issues_count":3,"forks_count":19,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-28T05:25:11.249Z","etag":null,"topics":["deep-learning","deep-neural-networks","deepfake-detection-challenge","deepfakes","deepfakes-classification","efficientnet","ensemble-learning","faceforensics","fake-news","pytorch","siamese-network","transfer-learning","triplet-loss","vision-transformers"],"latest_commit_sha":null,"homepage":"https://colab.research.google.com/drive/1s0e0OO_Xcyw7S81s8GydTDtTQXJvJPpL?usp=sharing","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chinmaynehate.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-11-28T21:14:51.000Z","updated_at":"2025-03-12T07:03:26.000Z","dependencies_parsed_at":"2023-01-20T02:04:22.676Z","dependency_job_id":null,"html_url":"https://github.com/chinmaynehate/DFSpot-Deepfake-Recognition","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chinmaynehate%2FDFSpot-Deepfake-Recognition","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chinmaynehate%2FDFSpot-Deepfake-Recognition/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chinmaynehate%2FDFSpot-Deepfake-Recognition/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chinmaynehate%2FDFSpot-Deepfake-Recognition/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chinmaynehate","download_url":"https://codeload.github.com/chinmaynehate/DFSpot-Deepfake-Recognition/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248918344,"owners_count":21183171,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deep-neural-networks","deepfake-detection-challenge","deepfakes","deepfakes-classification","efficientnet","ensemble-learning","faceforensics","fake-news","pytorch","siamese-network","transfer-learning","triplet-loss","vision-transformers"],"created_at":"2024-12-11T07:16:37.836Z","updated_at":"2025-10-04T21:05:38.563Z","avatar_url":"https://github.com/chinmaynehate.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv id=\"top\"\u003e\u003c/div\u003e\n\n\u003c!-- PROJECT SHIELDS --\u003e\n\n\n[![PR](https://img.shields.io/badge/PRs-Welcome-\u003cCOLOR\u003e.svg)][pullreq-url]\n[![Maintenance](https://img.shields.io/badge/Maintained%3F-Yes-\u003cCOLOR\u003e.svg)](https://https://github.com/chinmaynehate/DeepFake-Spot)\n[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/chinmaynehate/DeepFake-Spot/blob/master/LICENSE)\n[![Made with](https://img.shields.io/badge/Made%20with-Python-\u003cCOLOR\u003e.svg)](https://www.python.org/)\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1s0e0OO_Xcyw7S81s8GydTDtTQXJvJPpL?usp=sharing)\n[![PyTorch](https://img.shields.io/badge/Uses-PyTorch-\u003cCOLOR\u003e.svg)](https://pytorch.org/)\n\n\n\n\u003c!-- PROJECT LOGO --\u003e\n\u003cbr /\u003e\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://github.com/chinmaynehate/DeepFake-Spot\"\u003e\n    \u003cimg src=\"https://i.imgur.com/BhxJxjh.jpg\" alt=\"Logo\" \u003e\n  \u003c/a\u003e\n\n\u003ch2 align=\"center\"\u003eDFSpot-Deepfake-Recognition\u003c/h2\u003e\n\n  \u003cp align=\"center\"\u003e\n    Determine whether a given video sequence has been manipulated or synthetically generated\n    \u003cbr /\u003e\n    \u003ca href=\"https://github.com/chinmaynehate/DeepFake-Spot/issues\"\u003eReport Bug\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/chinmaynehate/DeepFake-Spot/issues\"\u003eRequest Feature\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/div\u003e\n\n\u003ch3 align=\"center\"\u003e⚡️ Try the demo here ⚡️\u003c/h3\u003e\n\u003cdiv align=\"center\"\u003e\n\n  \u003ca href=\"https://colab.research.google.com/drive/1s0e0OO_Xcyw7S81s8GydTDtTQXJvJPpL?usp=sharing\"\u003e![example1](https://colab.research.google.com/assets/colab-badge.svg)\u003c/a\u003e\n\n\u003c/div\u003e\n\n\n\u003ctable \u003e\n\u003cthead\u003e\n  \u003ctr\u003e\n    \u003cth \"\u003e\u003cimg src=\"assets/gifs/celeb_fake.gif\" alt=\"drawing\" width=\"600\" height=\"300\"/\u003e \u003c/th\u003e\n    \u003cth\u003e\u003cimg src=\"assets/gifs/ffpp_fake.gif\" alt=\"drawing\" width=\"600\" height=\"300\"/\u003e\u003c/th\u003e\n  \u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n  \n  \u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003cdiv align=\"center\"\u003e                                                                                   \n    \u003cstrong \u003eEnsemble of 4 models produce the above results on test videos from datasets such as Celeb-DF(v2), FaceForensics++ and DFDC \u003c/strong\u003e \n                   \u003cbr/\u003e\n\u003c/div\u003e                                                                               \n                                                                                   \n\u003cbr/\u003e                                                                                   \n\u003c!-- TABLE OF CONTENTS --\u003e\n\u003cdetails\u003e\n  \u003csummary\u003eTable of Contents\u003c/summary\u003e\n  \u003col\u003e\n    \u003cli\u003e\n      \u003ca href=\"#about-the-project\"\u003eAbout The Project\u003c/a\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"#built-with\"\u003eBuilt With\u003c/a\u003e\u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\n      \u003ca href=\"#getting-started\"\u003eGetting Started\u003c/a\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"#prerequisites\"\u003ePrerequisites\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\u003ca href=\"#installation\"\u003eInstallation\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\u003ca href=\"#repo-file-structure\"\u003eRepo file structure\u003c/a\u003e\u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#usage\"\u003eUsage\u003c/a\u003e\u003c/li\u003e    \n    \u003cli\u003e\u003ca href=\"#contributing\"\u003eContributing\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#license\"\u003eLicense\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#acknowledgments\"\u003eAcknowledgments\u003c/a\u003e\u003c/li\u003e\n  \u003c/ol\u003e\n\u003c/details\u003e\n\n\n\n\u003c!-- ABOUT THE PROJECT --\u003e\n## 📖 About The Project\n\nPyTorch code for DF-Spot, a model ensemble that determines if an input video/image is real or fraudulent. To identify deepfakes, this study proposes an ensemble-based metric learning technique based on a siamese network architecture, in which four models are built beginning from a base network. This method has been validated using publicly available datasets such as Celeb-DF (v2), FaceForensics++, and DFDC.\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n\n\n### :bricks: Built With\n\n* [Python 3.6.9](https://www.python.org/)\n* [PyTorch 1.8.0](https://pytorch.org/)\n* [Timm 0.4.9](https://github.com/rwightman/pytorch-image-models)\n* [OpenCV 4.2](https://opencv.org/)\n* [Albumentations 0.5.2](https://albumentations.ai/)\n* [Blazeface](https://github.com/tensorflow/tfjs-models/tree/master/blazeface)\n\n\n\n\u003c!-- GETTING STARTED --\u003e\n## ⚡️ Getting Started\nSet up the project on your local machine by following the instructions below. You can also run the demo on Google Colab [here](https://colab.research.google.com/drive/1s0e0OO_Xcyw7S81s8GydTDtTQXJvJPpL?usp=sharing)\n### :heavy_check_mark: Prerequisites\n* Update system and install pip3\n   ```sh\n   sudo apt update\n   sudo apt -y install python3-pip\n   ```\n\n* Python virtual environment (optional)\n   ```sh\n   sudo apt install python3-venv   \n   ```\n\n\n### ⚙️ Installation\n\n1. Create a python virtual environment (optional)\n   ```sh\n   mkdir df_spot\n   cd df_spot\n   python3 -m venv df_spot_env\n   source df_spot_env/bin/activate\n   ```\n2. Clone the repo\n   ```sh\n   git clone https://github.com/chinmaynehate/DeepFake-Spot.git\n   ```\n3. Install dependencies\n   ```sh\n   cd DFSpot-Deepfake-Recognition\n   sudo chmod +x setup.sh\n   ```                                \n\u003e **🔔 Note**\nThe following command, which runs the setup.sh file, requires the `-m` parameter, which accepts either dfdc, celeb, ffpp, or all as inputs. If the flag -m is used with the option dfdc, setup.sh will download the models trained on the dfdc dataset. The models are currently saved on Google Drive and there appears to be a limit to the number of files that may be downloaded using the command-line utility tool `gdown`. As a result, it is possible that this limit has been reached and you are unable to download the models. If this occurs, try running the script again after 24 hours. You can also manually download the models by visiting the google drive link for the models from the `setup.sh` file.     \nDownloading the models manually is recommended.\n                                                            \n   ```\n   ./setup.sh -m \u003cdataset\u003e\n   ```\nFor eg. If you want to download models trained on dfdc dataset, then run:                                                            \n   ```\n   ./setup.sh -m dfdc\n   ```\n  The other options are: celeb, ffpp or all                              \n\n                                \n\n\n\n### :floppy_disk: Project file structure\nAfter running the requirements, prerequisites and installation scripts, the directory structure of 'DFSpot-Deepfake-Recognition/' is as follows\n```sh\n|-- assets # contains images \u0026 gifs for readme\n|-- examples.sh # contains example for running spot_deepfakes.py \n|-- models # contains twelve .pth files. These are downloaded using gdown and extracted in setup.sh\n|   |-- celeb_v2.pth\n|   |-- dfdc_v2st.pth\n|   |-- ffpp_v2.pth\n|-- README.md\n|-- requirements.txt\n|-- sample_images # contains sample images from test set of ffpp, celebdf \u0026 dfdc dataset. Save the images that have to be tested in this folder           \n|-- sample_output_videos # contains sample output videos that are obtained after running the code \n|-- sample_videos # contains all the sample videos downloaded using gdown and extracted in setup.sh. Save the video files that have to be tested in this folder\n|   |-- abc.mp4 # video whose authenticity has to be tested\n|   |-- pqr.mp4 # video whose authenticity has to be tested\n|-- setup.sh # downloads all the models, sample_videos and installs dependencies\n|-- src\n    |-- architectures # contains definitions of models\n    |-- blazeface # for face extraction\n    |-- ensemble_model.ipynb \n    |-- output # contains the annotated video files generated by running spot_deepfakes.py\n    |   |-- abc.avi # annotated video with frame-level predictions done by the ensemble of models for sample_videos/abc.mp4\n    |   |-- pqr.avi # annotated video with frame-level predictions done by the ensemble of models for sample_videos/pqr.mp4\n    |   |-- predictions.csv # final prediction class of abc.mp4 and pqr.mp4 i.e real or fake is stored as csv\n    |-- spot_deepfakes.py # main()\n    |-- utils # contains functions for extraction of faces from videos in sample_videos, loading models, ensemble of models and annotation\n```\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n                                \n\u003c!-- USAGE EXAMPLES --\u003e\n## ⚡️ Usage\n### :video_camera: For videos\n                                \n1. When `setup.sh` is executed, a few example videos from the test set of datasets such as DFDC, FFPP, and CelebDF(V2) are saved in `sample videos/` folder. Assume you run the `setup.sh` file with the -m flag option dfdc. If so, then pass dfdc as the `--dataset` argument, and the code will check for models trained on the dfdc dataset in the models directory specified by the `--model dir` argument. Command to check for deepfakes in these videos using models trained on dfdc dataset is:\n```sh\npython3 spot_deepfakes.py --media_type video --data_dir ../sample_videos/dfdc/fake/ --dataset dfdc --model TimmV2 TimmV2ST ViT ViTST  --model_dir ../models/ --video_id 2 3 4 --annotate True --device 0 --output_dir output/  \n```\nThe predictions are stored in `output/predictions.csv` and video with frame level annotations of predictions made by individual models and ensemble of models is stored in `output/` folder.\n\n2. Say you have three videos- video1.mp4, video2.mp4 and video3.mp4 and you want to check their authenticity. Place these three videos in the `sample_videos/` folder and run:\n```sh\npython3 spot_deepfakes.py --media_type video --data_dir ../sample_videos/ --dataset ffpp --model TimmV2 TimmV2ST ViT ViTST  --model_dir ../models/ --video_id 0 1 2 --annotate True --device 0 --output_dir output/  \n```\nThe predictions are stored in `output/predictions.csv` and video with frame level annotations of predictions made by individual models and ensemble of models is stored in `output/` folder.\n\n### :framed_picture: For images\n                                \n1. By running `setup.sh` during installation, few sample images from test set of datasets like DFDC, FFPP and CelebDF(V2) are saved in `sample_images/`. To check the authenticity of these images, run:\n```sh\npython3 spot_deepfakes.py --media_type image --data_dir ../sample_images/ --dataset dfdc --model TimmV2 TimmV2ST ViT ViTST --model_dir ../models  --device 0 --output_dir output/  \n```\n2. Say you have a few images and you need to check their authenticity. Place them in the `sample_images/` folder and run the following command:\n``` sh\npython3 spot_deepfakes.py --media_type image --data_dir ../sample_images/ --dataset dfdc --model TimmV2 TimmV2ST ViT ViTST --model_dir ../models  --device 0 --output_dir output/  \n```\n                                \nThe predictions are stored in `output/img_predictions.json`\n \n**_For more examples, please refer to [examples.sh](https://github.com/chinmaynehate/DeepFake-Spot/blob/master/examples.sh)_**\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n\n\n\n\u003c!-- CONTRIBUTING --\u003e\n## ⭐️ Contributing\n\nAny contributions you make are **greatly appreciated**.\n\nIf you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag \"enhancement\".\n\n1. Fork the Project\n2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the Branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n\u003c!-- LICENSE --\u003e\n## ⚠️ License\n\nDistributed under the MIT License. See [`LICENSE`](https://github.com/chinmaynehate/DFSpot-Deepfake-Recognition/blob/master/LICENSE) for more information.\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n                                \n\u003c!-- ACKNOWLEDGMENTS --\u003e\n## :thumbsup: Acknowledgments\n\n* [Deepware](https://github.com/deepware/deepfake-scanner)\n* [Image and Sound Processing Lab - Politecnico di Milano](https://github.com/polimi-ispl/icpr2020dfdc)\n* [Triplet loss tutorial](https://omoindrot.github.io/triplet-loss)\n* Facebook's DeepFake Detection Challenge (DFDC) dataset [Paper](https://arxiv.org/abs/2006.07397)\n* FaceForensics++ [Paper](https://arxiv.org/abs/1901.08971)\n* Celeb-DF (v2) [Paper](https://arxiv.org/abs/1909.12962) \n* [Other papers](https://docs.google.com/document/d/1vW7ksUl4JFrQwUJIJ63SSmxxsMK84iVyGbEmjOCh__c/edit?usp=sharing)\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n## :star: How to cite\nPlain text:\n```\nC. Nehate, P. Dalia, S. Naik and A. Bhan, \"Exposing DeepFakes using Siamese Training,\" 2022 IEEE India Council International Subsections Conference (INDISCON), 2022, pp. 1-6, doi: 10.1109/INDISCON54605.2022.9862825.\n```\n\nBibtex:\n```bibtex\n@INPROCEEDINGS{9862825,\n  author={Nehate, Chinmay and Dalia, Parth and Naik, Saket and Bhan, Aditya},\n  booktitle={2022 IEEE India Council International Subsections Conference (INDISCON)}, \n  title={Exposing DeepFakes using Siamese Training}, \n  year={2022},\n  volume={},\n  number={},\n  pages={1-6},\n  doi={10.1109/INDISCON54605.2022.9862825}}\n```\n\n\u003c!-- MARKDOWN LINKS \u0026 IMAGES --\u003e\n\u003c!-- https://www.markdownguide.org/basic-syntax/#reference-style-links --\u003e\n\n[pullreq-url]:https://github.com/chinmaynehate/DeepFake-Spot/pulls\n[contributors-shield]: https://img.shields.io/github/contributors/github_username/repo_name.svg?style=for-the-badge\n[contributors-url]: https://github.com/github_username/repo_name/graphs/contributors\n[forks-shield]: https://img.shields.io/github/forks/github_username/repo_name.svg?style=for-the-badge\n[forks-url]: https://github.com/github_username/repo_name/network/members\n[stars-shield]: https://img.shields.io/github/stars/github_username/repo_name.svg?style=for-the-badge\n[stars-url]: https://github.com/github_username/repo_name/stargazers\n[issues-shield]: https://img.shields.io/github/issues/github_username/repo_name.svg?style=for-the-badge\n[issues-url]: https://github.com/github_username/repo_name/issues\n[license-shield]: https://img.shields.io/github/license/github_username/repo_name.svg?style=for-the-badge\n[license-url]: https://github.com/github_username/repo_name/blob/master/LICENSE.txt\n[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge\u0026logo=linkedin\u0026colorB=555\n[linkedin-url]: https://linkedin.com/in/linkedin_username\n[product-screenshot]: images/screenshot.png\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchinmaynehate%2Fdfspot-deepfake-recognition","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchinmaynehate%2Fdfspot-deepfake-recognition","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchinmaynehate%2Fdfspot-deepfake-recognition/lists"}