{"id":27373418,"url":"https://github.com/zakircodearchitect/sonic-lipsync-ai","last_synced_at":"2026-03-02T14:33:19.477Z","repository":{"id":286201188,"uuid":"957412897","full_name":"ZakirCodeArchitect/Sonic-Lipsync-AI","owner":"ZakirCodeArchitect","description":"A Google Colab-based Gradio app for generating lip-synced videos using the Sonic model. It supports audio-to-video syncing with Hugging Face models and runs entirely in the cloud—no local setup needed.","archived":false,"fork":false,"pushed_at":"2025-04-04T22:57:18.000Z","size":8848,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-13T11:14:41.768Z","etag":null,"topics":["deep-learning","diffusers","gradio","huggingface","lipsync","openai-whisper","python","pytorch","sonic-model"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ZakirCodeArchitect.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-30T10:03:43.000Z","updated_at":"2025-04-08T11:34:35.000Z","dependencies_parsed_at":"2025-04-04T23:36:04.163Z","dependency_job_id":null,"html_url":"https://github.com/ZakirCodeArchitect/Sonic-Lipsync-AI","commit_stats":null,"previous_names":["zakircodearchitect/sonic-lipsync-ai"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ZakirCodeArchitect/Sonic-Lipsync-AI","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZakirCodeArchitect%2FSonic-Lipsync-AI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZakirCodeArchitect%2FSonic-Lipsync-AI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZakirCodeArchitect%2FSonic-Lipsync-AI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZakirCodeArchitect%2FSonic-Lipsync-AI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ZakirCodeArchitect","download_url":"https://codeload.github.com/ZakirCodeArchitect/Sonic-Lipsync-AI/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZakirCodeArchitect%2FSonic-Lipsync-AI/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266135685,"owners_count":23881803,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","diffusers","gradio","huggingface","lipsync","openai-whisper","python","pytorch","sonic-model"],"created_at":"2025-04-13T11:14:40.020Z","updated_at":"2026-03-02T14:33:19.389Z","avatar_url":"https://github.com/ZakirCodeArchitect.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🎤 Sonic Lip Sync - Gradio App (Google Colab Setup)\n\nThis project brings the [Sonic](https://github.com/jixiaozhong/Sonic) lip-sync model into a user-friendly Gradio interface, allowing you to generate realistic lip-synced videos directly in **Google Colab** using Hugging Face models. No local installation required.\n\n---\n\n## 🚀 Demo\nLaunches a `.gradio.live` URL from Colab to interact with the app in your browser.\n\n---\n\n## 🧰 Features\n- 🔊 Converts images and audio into lip-synced video\n- 🤖 Powered by: `Sonic`, `Stable Video Diffusion`, `Whisper-tiny`\n- 🌐 Hosted in Google Colab with full CUDA and PyTorch support\n- 🧠 Uses Hugging Face CLI to fetch model checkpoints\n\n---\n\n## ⚙️ Setup Instructions\n\n### 🗂️ 1. Clone the Project in Colab\n```bash\nmkdir ai-tools\ncd ai-tools\nmkdir sonic-lip-sync\ncd sonic-lip-sync\ngit clone https://github.com/jixiaozhong/Sonic.git\ncd Sonic\n```\n\n### 🧪 2. Create a Virtual Environment (Not required if using Colab or any hosted Environement)\n```bash\npython -m venv venv\ncall venv\\scripts\\activate\n```\n\n### ⚙️ 3. Install CUDA Toolkit 12.6 (recommended version 12.6)\n```bash\nconda install nvidia/label/cuda-12.6.2::cuda-toolkit\n```\n\n### 📄 4. View and update the requirements.txt\n```bash\ncat requirements.txt\n```\n- remove the torch commands (total 3)\n  \n\n### 🔥 5. Install PyTorch with CUDA 12.6\n```bash\npip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126\n```\n\n### 📦 6. Install the requirements.txt\n```bash\npip install -r requirments.txt\n```\n\n### 🤗 7. Install and download Hugging Face CLI Models\n```bash\npython -m pip install \"huggingface_hub[cli]\"\n\nhuggingface-cli download LeonJoe13/Sonic --local-dir checkpoints\nhuggingface-cli download stabilityai/stable-video-diffusion-img2vid-xt --local-dir checkpoints/stable-video-diffusion-img2vid-xt\nhuggingface-cli download openai/whisper-tiny --local-dir checkpoints/whisper-tiny\n```\n\n### 🧩 8. Install remaining dependencies\n```bash\npip install opencv-python\npip install accelerate\n```\n\n### 📝 9. Open and Update the gradio_app.py\n- Go to the end of the file.\n- Replace the server from 0.0.0.0 to 127.0.0.1\n\n### ▶️ 10. Run the Application\n```bash\npython gradio_app.py\n```\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzakircodearchitect%2Fsonic-lipsync-ai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzakircodearchitect%2Fsonic-lipsync-ai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzakircodearchitect%2Fsonic-lipsync-ai/lists"}