{"id":13627075,"url":"https://github.com/datitran/face2face-demo","last_synced_at":"2025-04-08T10:13:59.241Z","repository":{"id":38355952,"uuid":"96015791","full_name":"datitran/face2face-demo","owner":"datitran","description":"pix2pix demo that learns from facial landmarks and translates this into a face","archived":false,"fork":false,"pushed_at":"2023-10-10T18:29:41.000Z","size":7655,"stargazers_count":1447,"open_issues_count":29,"forks_count":421,"subscribers_count":72,"default_branch":"master","last_synced_at":"2025-04-01T09:19:45.512Z","etag":null,"topics":["pix2pix-tensorflow","python3","tensorflow"],"latest_commit_sha":null,"homepage":"https://medium.com/@datitran/face2face-a-pix2pix-demo-that-mimics-the-facial-expression-of-the-german-chancellor-b6771d65bf66","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datitran.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-07-02T10:18:00.000Z","updated_at":"2025-03-28T07:59:28.000Z","dependencies_parsed_at":"2024-01-14T06:07:13.888Z","dependency_job_id":"0a903ed1-f39d-4ec5-a6df-7a08e62836c1","html_url":"https://github.com/datitran/face2face-demo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datitran%2Fface2face-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datitran%2Fface2face-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datitran%2Fface2face-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datitran%2Fface2face-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datitran","download_url":"https://codeload.github.com/datitran/face2face-demo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247819933,"owners_count":21001394,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["pix2pix-tensorflow","python3","tensorflow"],"created_at":"2024-08-01T22:00:30.073Z","updated_at":"2025-04-08T10:13:59.201Z","avatar_url":"https://github.com/datitran.png","language":"Python","funding_links":[],"categories":["Python","Tools","5. Generation of synthetic content","Deepfake Videos and Images"],"sub_categories":["Image Datasets","5.4 Generation Videos","Generation Codes"],"readme":"# face2face-demo\n\nThis is a pix2pix demo that learns from facial landmarks and translates this into a face. A webcam-enabled application is also provided that translates your face to the trained face in real-time.\n\n## Getting Started\n\n#### 1. Prepare Environment\n\n```\n# Clone this repo\ngit clone git@github.com:datitran/face2face-demo.git\n\n# Create the conda environment from file (Mac OSX)\nconda env create -f environment.yml\n```\n\n#### 2. Generate Training Data\n\n```\npython generate_train_data.py --file angela_merkel_speech.mp4 --num 400 --landmark-model shape_predictor_68_face_landmarks.dat\n```\n\nInput:\n\n- `file` is the name of the video file from which you want to create the data set.\n- `num` is the number of train data to be created.\n- `landmark-model` is the facial landmark model that is used to detect the landmarks. A pre-trained facial landmark model is provided [here](http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2).\n\nOutput:\n\n- Two folders `original` and `landmarks` will be created.\n\nIf you want to download my dataset, here is also the [video file](https://dl.dropboxusercontent.com/s/2g04onlkmkq9c69/angela_merkel_speech.mp4) that I used and the generated [training dataset](https://dl.dropboxusercontent.com/s/pfm8b0yogmum63w/dataset.zip) (400 images already split into training and validation).\n\n#### 3. Train Model\n\n```\n# Clone the repo from Christopher Hesse's pix2pix TensorFlow implementation\ngit clone https://github.com/affinelayer/pix2pix-tensorflow.git\n\n# Move the original and landmarks folder into the pix2pix-tensorflow folder\nmv face2face-demo/landmarks face2face-demo/original pix2pix-tensorflow/photos\n\n# Go into the pix2pix-tensorflow folder\ncd pix2pix-tensorflow/\n\n# Resize original images\npython tools/process.py \\\n  --input_dir photos/original \\\n  --operation resize \\\n  --output_dir photos/original_resized\n  \n# Resize landmark images\npython tools/process.py \\\n  --input_dir photos/landmarks \\\n  --operation resize \\\n  --output_dir photos/landmarks_resized\n  \n# Combine both resized original and landmark images\npython tools/process.py \\\n  --input_dir photos/landmarks_resized \\\n  --b_dir photos/original_resized \\\n  --operation combine \\\n  --output_dir photos/combined\n  \n# Split into train/val set\npython tools/split.py \\\n  --dir photos/combined\n  \n# Train the model on the data\npython pix2pix.py \\\n  --mode train \\\n  --output_dir face2face-model \\\n  --max_epochs 200 \\\n  --input_dir photos/combined/train \\\n  --which_direction AtoB\n```\n\nFor more information around training, have a look at Christopher Hesse's [pix2pix-tensorflow](https://github.com/affinelayer/pix2pix-tensorflow) implementation.\n\n#### 4. Export Model\n\n1. First, we need to reduce the trained model so that we can use an image tensor as input: \n    ```\n    python reduce_model.py --model-input face2face-model --model-output face2face-reduced-model\n    ```\n    \n    Input:\n    \n    - `model-input` is the model folder to be imported.\n    - `model-output` is the model (reduced) folder to be exported.\n    \n    Output:\n    \n    - It returns a reduced model with less weights file size than the original model.\n\n2. Second, we freeze the reduced model to a single file.\n    ```\n    python freeze_model.py --model-folder face2face-reduced-model\n    ```\n\n    Input:\n    \n    - `model-folder` is the model folder of the reduced model.\n    \n    Output:\n    \n    - It returns a frozen model file `frozen_model.pb` in the model folder.\n    \nI have uploaded a pre-trained frozen model [here](https://dl.dropboxusercontent.com/s/rzfaoeb3e2ta343/face2face_model_epoch_200.zip). This model is trained on 400 images with epoch 200.\n    \n#### 5. Run Demo\n\n```\npython run_webcam.py --source 0 --show 0 --landmark-model shape_predictor_68_face_landmarks.dat --tf-model face2face-reduced-model/frozen_model.pb\n```\n\nInput:\n\n- `source` is the device index of the camera (default=0).\n- `show` is an option to either display the normal input (0) or the facial landmark (1) alongside the generated image (default=0).\n- `landmark-model` is the facial landmark model that is used to detect the landmarks.\n- `tf-model` is the frozen model file.\n\nExample:\n\n![example](example.gif)\n\n## Requirements\n- [Anaconda / Python 3.5](https://www.continuum.io/downloads)\n- [TensorFlow 1.2](https://www.tensorflow.org/)\n- [OpenCV 3.0](http://opencv.org/)\n- [Dlib 19.4](http://dlib.net/)\n\n## Acknowledgments\nKudos to [Christopher Hesse](https://github.com/christopherhesse) for his amazing pix2pix TensorFlow implementation and [Gene Kogan](http://genekogan.com/) for his inspirational workshop. \n\n## Copyright\n\nSee [LICENSE](LICENSE) for details.\nCopyright (c) 2017 [Dat Tran](http://www.dat-tran.com/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatitran%2Fface2face-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatitran%2Fface2face-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatitran%2Fface2face-demo/lists"}