https://github.com/parkyunsu/voicetolipsync

Voice to LipSync : High-Quality Lip Sync Video Genreation OpenVoice to Generative a Zero-Shot TTS and Wav2Lip to Generative Lip Sync Video.
https://github.com/parkyunsu/voicetolipsync

openvoice wav2lip

Last synced: 3 months ago
JSON representation

Voice to LipSync : High-Quality Lip Sync Video Genreation OpenVoice to Generative a Zero-Shot TTS and Wav2Lip to Generative Lip Sync Video.

• Host: GitHub
• URL: https://github.com/parkyunsu/voicetolipsync
• Owner: PARKYUNSU
• Created: 2025-01-15T05:54:37.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2025-02-01T08:32:30.000Z (4 months ago)
• Last Synced: 2025-02-01T09:25:50.376Z (4 months ago)
• Topics: openvoice, wav2lip
• Language: Python
• Homepage:
• Size: 34.2 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# VoiceToLipSync: High-Quality Lip Sync Video Generation

PyTorch implementation of LipSync-Voice: High-Quality Lip Sync Video Generation using Wav2Lip & Openvoice models.



  



### Tech Stack



  

  

  

  

  

  



---

LipSync-Voice is a high-quality lip-sync video generation system that leverages deep learning to synchronize facial movements with speech. This framework processes input a sample video through ``LipSync.ipynb``, extracting speech features and synchronizing them with facial motions to create realistic lip-sync videos—all without requiring manual installation of dependencies.

## Project Duration

**2025.01.01 - 2025.01.20**

  

    

      

        

          

          
Yunsu Park

        

        


      

        

          

          
Myoungjin Son

        

        


      

    

  

## Presentation

The presentation deck is available in the `deck` folder: [LipSync_Voice_Presentation.pdf](https://github.com/PARKYUNSU/LipSync-Voice/blob/main/deck/LipSync_Voice_Presentation.pdf).

## How to Use

1. Click the "Open in Colab" button below.

2. Follow the instructions in the notebook to upload your video files.

3. Generate the lip-sync video and download the result.

## VoiceToLipSync Demo

Click the badge below to run the demo

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/PARKYUNSU/VoiceToLipSync/blob/main/Voice2LipSync.ipynb)

## Results

### Reference Video

https://github.com/user-attachments/assets/8e48cdc3-eb0b-4e3c-9cb2-222f8edfd3cd

### Lip Synced Video

https://github.com/user-attachments/assets/ffc0ac9f-9944-4ff9-aa19-3139182f5249

## References

### Lip Syncing

- **Wav2Lip**: A model for lip-syncing in videos, where the facial movements are synchronized with the audio.  

  Repository: [https://github.com/Rudrabha/Wav2Lip](https://github.com/Rudrabha/Wav2Lip)

### Speech Synthesis and Voice Conversion

- **OpenVoice**: A deep learning framework for text-to-speech conversion with high-quality voice synthesis.  

  Repository: [https://github.com/OpenVoice](https://github.com/OpenVoice)

---