{"id":26068268,"url":"https://github.com/fralapo/lipsyncify","last_synced_at":"2026-04-29T21:35:51.745Z","repository":{"id":281344180,"uuid":"944993054","full_name":"fralapo/LipSyncify","owner":"fralapo","description":"Automatically create lip-sync videos from audio using Rhubarb Lip Sync, Whisper AI transcription, and FFmpeg. Supports MP4 with custom backgrounds and MOV with transparency. Easily customizable mouth images, GPU acceleration, and simple setup. 🚀","archived":false,"fork":false,"pushed_at":"2025-03-08T12:39:02.000Z","size":238,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-08T13:19:31.561Z","etag":null,"topics":["animation","ffmpeg","lipsync","transcription","whisper"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fralapo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-08T12:05:14.000Z","updated_at":"2025-03-08T12:39:05.000Z","dependencies_parsed_at":"2025-03-08T13:29:36.574Z","dependency_job_id":null,"html_url":"https://github.com/fralapo/LipSyncify","commit_stats":null,"previous_names":["fralapo/lipsync-generator-with-rhubarb-and-whisper-ai"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fralapo%2FLipSyncify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fralapo%2FLipSyncify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fralapo%2FLipSyncify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fralapo%2FLipSyncify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fralapo","download_url":"https://codeload.github.com/fralapo/LipSyncify/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242615620,"owners_count":20158471,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["animation","ffmpeg","lipsync","transcription","whisper"],"created_at":"2025-03-08T22:22:27.154Z","updated_at":"2026-04-29T21:35:51.721Z","avatar_url":"https://github.com/fralapo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🎬 Lipsync Generator: Rhubarb \u0026 Whisper AI 🎙️\n\nQuickly generate videos with lip synchronization by combining voice audio and custom mouth images.\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"logo.jpeg\" alt=\"Lipsync Generator Logo\" width=\"120\"\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003cstrong\u003eRhubarb Lip Sync\u003c/strong\u003e automatically calculates the timing of lip movements.\u003c/li\u003e\n        \u003cli\u003e\u003cstrong\u003eWhisper AI\u003c/strong\u003e automatically generates audio transcription, if needed.\u003c/li\u003e\n        \u003cli\u003e\u003cstrong\u003eFFmpeg\u003c/strong\u003e exports the final result in \u003ccode\u003e.mp4\u003c/code\u003e video with custom background or transparent \u003ccode\u003e.mov\u003c/code\u003e.\u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n---\n\n## 🛠️ Installation\n\n### 1️⃣ Download Rhubarb Lip Sync\nDownload the latest release from the official repository:\n- 👉 [Rhubarb Lip Sync Releases](https://github.com/DanielSWolf/rhubarb-lip-sync/releases)\n\nSelect the file compatible with your operating system:\n- **Windows:** `Rhubarb-Lip-Sync-\u003cversion\u003e-Windows.zip`\n- **Linux:** `Rhubarb-Lip-Sync-\u003cversion\u003e-Linux.zip`\n- **macOS:** `Rhubarb-Lip-Sync-\u003cversion\u003e-macOS.zip`\n\n📌 Extract the contents into a dedicated folder.\n\n### 2️⃣ Clone this repository\nOpen a terminal in the newly extracted folder and run:\n```bash\ncd Rhubarb-Lip-Sync-\u003cversion\u003e-\u003csystem\u003e  # Change with the correct name of your folder\ngit clone https://github.com/fralapo/LipSyncify.git\n```\n\n### 3️⃣ Install Python dependencies\nMake sure Python is installed (recommended version: 3.10 or higher):\n```bash\npip install -r requirements.txt\n```\n\n📌 For GPU support with Whisper (optional but recommended):\n```bash\npip install torch torchaudio torchvision --extra-index-url https://download.pytorch.org/whl/cu118\n```\n\n### 4️⃣ Install FFmpeg\nCheck if FFmpeg is already installed:\n```bash\nffmpeg -version\n```\nIf needed, download it from [here](https://ffmpeg.org/download.html).\n\n---\n\n## 📂 Project structure\n```\n📁 Rhubarb-Lip-Sync/\n├── 📁 assets/\n│   ├── 📁 mouth_images/      # Transparent PNGs for mouths\n│   ├── 🎵 audio.wav          # Audio file to synchronize\n│   └── 📜 transcript.txt     # (Optional) Audio transcription\n├── 📝 generate_lipsync.py    # Main script\n├── 📜 requirements.txt       # Python dependencies\n└── 🛠 rhubarb                 # Rhubarb Lip Sync executable\n```\n\n📌 If the `transcript.txt` file is missing, Whisper will generate it automatically.\n\n---\n\n## 🚀 How to use it\n\nOpen the terminal in the project folder and run:\n\nUsage examples:\n```bash\npython3 generate_lipsync.py                         # Generate MP4 with white background\n```\n```bash\npython3 generate_lipsync.py --background yellow     # Generate MP4 with yellow background\n```\n```bash\npython3 generate_lipsync.py --format mov            # Generate MOV with transparency\n```\n```bash\npython3 generate_lipsync.py --model small --cpu     # Use small Whisper model on CPU\n```\n```bash\npython3 generate_lipsync.py --help                  # Show this help message\n```\n\nAvailable colors:\n```\nwhite, black, red, green, blue, yellow, cyan, magenta, purple,\norange, pink, brown, gold, silver, navy, teal, olive, maroon,\ngray/grey, lime\n```\n\nYou can also specify hexadecimal colors:\n```bash\npython3 generate_lipsync.py --background 00FF00     # Green (hexadecimal)\n```\n```bash\npython3 generate_lipsync.py --background #FF0000    # Red (hexadecimal with #)\n```\n\n---\n\n## 🎨 Customizing mouth images\n\nYou can change the images by inserting your PNG files in the folder:\n```\nassets/mouth_images/\n```\n\n| Viseme | Description | PNG file name |\n|--------|-------------|-------------------|\n| A | Closed mouth for M, B, P | `mouth_A.png` |\n| B | Semi-open mouth for K, S, T | `mouth_B.png` |\n| C | Open mouth for E, AE | `mouth_C.png` |\n| D | Wide mouth for AA, O | `mouth_D.png` |\n| E | Round mouth for O, ER | `mouth_E.png` |\n| F | Narrow mouth for UW, W | `mouth_F.png` |\n| G | Teeth over lip for F, V | `mouth_G.png` |\n| H | Visible tongue for L | `mouth_H.png` |\n| X | Resting mouth | `mouth_X.png` |\n\n---\n\n## ❓ Frequently Asked Questions\n\n- **How to check if FFmpeg is installed?**\n  ```bash\n  ffmpeg -version\n  ```\n\n- **Transparent background not working?**\n  - Use a compatible video player (e.g., QuickTime, Adobe Premiere, DaVinci Resolve).\n\n- **Problems with Whisper and GPU?**\n  - Use the `--cpu` option or install PyTorch compatible with your GPU.\n\n---\n\n## 🙏 **Acknowledgments**\n\nSpecial thanks to the following open-source projects that made this repository possible:\n\n- [**OpenAI Whisper**](https://github.com/openai/whisper) – for powerful and accurate audio transcription.\n- [**Rhubarb Lip Sync**](https://github.com/DanielSWolf/rhubarb-lip-sync) – a robust tool for automated lip synchronization.\n\n---\n\n🎉 **Happy lipsync!** 🚀\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffralapo%2Flipsyncify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffralapo%2Flipsyncify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffralapo%2Flipsyncify/lists"}