{"id":22852413,"url":"https://github.com/abdur75648/dinet-inference","last_synced_at":"2025-03-31T07:09:48.964Z","repository":{"id":246110988,"uuid":"820126175","full_name":"abdur75648/DINet-Inference","owner":"abdur75648","description":"Create high-resolution visually dubbed videos with DINet","archived":false,"fork":false,"pushed_at":"2024-06-27T16:30:42.000Z","size":47,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-06T11:37:32.438Z","etag":null,"topics":["avatar-generation","deepspeech","dinet","dubbing","lipgan","lipsync","openface","video-generation","wav2l","wav2vec"],"latest_commit_sha":null,"homepage":"https://github.com/abdur75648/DINet-Inference","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/abdur75648.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-25T21:20:37.000Z","updated_at":"2024-07-03T07:12:09.000Z","dependencies_parsed_at":"2024-06-25T22:48:22.263Z","dependency_job_id":"a16a7d19-9884-4f81-9e1e-a00086c5d70f","html_url":"https://github.com/abdur75648/DINet-Inference","commit_stats":null,"previous_names":["abdur75648/dinet-inference"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdur75648%2FDINet-Inference","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdur75648%2FDINet-Inference/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdur75648%2FDINet-Inference/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdur75648%2FDINet-Inference/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/abdur75648","download_url":"https://codeload.github.com/abdur75648/DINet-Inference/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246429487,"owners_count":20775807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["avatar-generation","deepspeech","dinet","dubbing","lipgan","lipsync","openface","video-generation","wav2l","wav2vec"],"created_at":"2024-12-13T06:08:14.649Z","updated_at":"2025-03-31T07:09:48.936Z","avatar_url":"https://github.com/abdur75648.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Video (AAAI2023)\n\nA well-documented version of the DINet model for inference on custom videos.\nThis repo provides a step-by-step guide to run the DINet model either on a local machine or on Google Colab.\n\n## Evironment Setup\n1. Clone this repository:\n```bash\nhttps://github.com/abdur75648/DINet.git\ncd DINet\n```\n\n2. Create a conda environment with python==3.7 and install all dependencies:\n```bash\nconda create -n dinet python==3.7 -y\nconda activate dinet\npip install -r requirements.txt\n```\n*If running in Google Colab, you will first need to install Miniconda and then install the packages in the requirements.txt file. The steps are as follows:*\n```bash\n!wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh\n!chmod +x Miniconda3-py37_4.8.2-Linux-x86_64.sh\n!bash ./Miniconda3-py37_4.8.2-Linux-x86_64.sh -b -f -p /usr/local\nimport sys\nsys.path.append('/usr/local/lib/python3.8/site-packages/')\n!conda install python=3.7 -y\n!python --version\n!conda create --name dinet python=3.7 -y\n!source activate dinet\n!pip install -r requirements.txt\n```\n\n3. Download the model and sample data (asserts.zip) in [Google drive](https://drive.google.com/uc\\?id\\=1CkeEn7l3PuubuJIMWNjWpIrt0HDd_AB3), unzip and put it in the root directory.\n```bash\ngdown --id 1CkeEn7l3PuubuJIMWNjWpIrt0HDd_AB3\nunzip asserts.zip\nrm asserts.zip\n```\n\n## Inference\n* For running inference on the example videos, run the following command:\n```bash\npython inference.py --mouth_region_size 256 --source_video_path test.mp4 --source_openface_landmark_path test.csv --driving_audio_path test.wav --res_video_dir test_output/ --pretrained_clip_DINet_path ./asserts/clip_training_DINet_256mouth.pth\n```\n\n## Additional Setup Notes\n### OpenFace\n\nThe `.csv` file should contain the facial landmarks detected using [OpenFace](https://github.com/TadasBaltrusaitis/OpenFace). Installing OpenFace from scratch can be challenging due to dependency issues. Therefore, it is recommended to use Docker for a quicker setup. Follow these steps for quickstart usage of OpenFace with Docker:\n\n1. Run the OpenFace Docker container:\n    ```bash\n    docker run -it --rm algebr/openface:latest\n    ```\n2. Find the container ID by running this (*in a different terminal*):\n    ```bash\n    docker ps\n    ```\n    (Let's say it shows `a52fea727822`)\n\n3. Transfer any video you want to run OpenFace on:\n    ```bash\n    docker cp test.mp4 a52fea727822:/home/openface-build\n    ```\n4. In the first shell (*where the container is running*), run OpenFace on the video:\n    ```bash\n    build/bin/FaceLandmarkVidMulti -f test.mp4 -2Dfp\n    ```\n5. The output will be saved as `test.csv` in the `processed` directory. Transfer it back to your local machine (*in a different terminal*):\n    ```bash\n    docker cp a52fea727822:/home/openface-build/processed/test.csv .\n    ```\n\n* Note that we are using `FaceLandmarkVidMulti` here for videos, than `FaceLandmarkVid` (For images, use `FaceLandmarkImg`). This is because `FaceLandmarkVid` requires a display for its operations, which is not available in a Docker container by default. If you need to use `FaceLandmarkVid`, you can set up X11 forwarding on your local machine to enable the display (Ref. [this issue](https://github.com/TadasBaltrusaitis/OpenFace/issues/1076#issuecomment-2192538076)).\n\n### FFMPEG\nEnsure that FFMPEG is installed on your system to enable audio and video merging functionality in the DINet model. \n\nTo install FFMPEG, if you have root access to your system, run the following command:\n\n    ```bash\n    sudo apt-get install ffmpeg\n    ```\n\nIf you don't have root access, follow the instructions below to install FFMPEG statically in your root directory:\n1. **Download the Correct Static Build For Your System:** (Architecture can be checked using ```uname -m``` command)\n    ```bash\n    # For i686:\n    wget -O ffmpeg.tar.xz https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-i686-static.tar.xz\n    # For x86_64 or amd64:\n    wget -O ffmpeg.tar.xz https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz\n    # For arm64 or aarch64:\n    wget -O ffmpeg.tar.xz https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-arm64-static.tar.xz\n    ```\n\n2. **Unzip and Unpack the Build:**\n    ```bash\n    tar xvf ffmpeg.tar.xz\n    rm ffmpeg.tar.xz\n    ```\n\n4. **Go to the unzipped directory:** (Naming convention is ```ffmpeg-git-[YYYYMMDD]-[platform]-static```)\n    ```bash\n    cd ffmpeg-git-*-static\n    ```\n\n3. **Check the Installation:**\n    ```bash\n    ./ffmpeg -version\n    ```\n\n4. **Move the Binary to the root Directory:**\n    ```bash\n    cd ..\n    mv ffmpeg DINet/\n    ```\n\n## Acknowledge\nThis code is taken from the original repository of [DINet](https://github.com/MRzzm/DINet)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabdur75648%2Fdinet-inference","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabdur75648%2Fdinet-inference","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabdur75648%2Fdinet-inference/lists"}