{"id":20314750,"url":"https://github.com/mazzasaverio/youtube-auto-dub","last_synced_at":"2025-09-22T22:31:25.265Z","repository":{"id":220798595,"uuid":"752275753","full_name":"mazzasaverio/youtube-auto-dub","owner":"mazzasaverio","description":"Automated voice dubbing for YouTube videos using Docker, OpenVoice, and FastAPI. Translates and dubs videos with original voice timbre.","archived":false,"fork":false,"pushed_at":"2024-02-09T17:04:29.000Z","size":10849,"stargazers_count":50,"open_issues_count":0,"forks_count":9,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-05T05:24:51.454Z","etag":null,"topics":["docker","fastapi","gcp","openvoice","serverless","terraform","text-to-speech","tts","voice-clone","youtube","zero-shot-tts"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mazzasaverio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-03T14:58:52.000Z","updated_at":"2025-03-22T18:41:50.000Z","dependencies_parsed_at":"2025-01-14T12:48:31.377Z","dependency_job_id":"395b5f9b-d873-44de-a5b1-7d1c53471592","html_url":"https://github.com/mazzasaverio/youtube-auto-dub","commit_stats":null,"previous_names":["mazzasaverio/youtube-auto-dub"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mazzasaverio/youtube-auto-dub","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzasaverio%2Fyoutube-auto-dub","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzasaverio%2Fyoutube-auto-dub/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzasaverio%2Fyoutube-auto-dub/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzasaverio%2Fyoutube-auto-dub/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mazzasaverio","download_url":"https://codeload.github.com/mazzasaverio/youtube-auto-dub/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzasaverio%2Fyoutube-auto-dub/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276487200,"owners_count":25651132,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-22T02:00:08.972Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","fastapi","gcp","openvoice","serverless","terraform","text-to-speech","tts","voice-clone","youtube","zero-shot-tts"],"created_at":"2024-11-14T18:16:35.211Z","updated_at":"2025-09-22T22:31:24.971Z","avatar_url":"https://github.com/mazzasaverio.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# YouTube Auto-Dub\n\nThis repository serves as a starting point for developing a FastAPI backend for dubbing YouTube videos by capturing and inferring the voice timbre using OpenVoice.\n\n![Example Image](static/screen.png)\n\n## Core Features\n\n- **Voice Timbre Recognition**: Utilizes OpenVoice technology to accurately recognize the voice timbre from the original YouTube video.\n- **Text-to-Speech Synthesis**: Downloads and processes subtitles, translating them and converting them into speech, matching the original voice timbre as closely as possible.\n- **Flexible Deployment**: Supports deployment via GitHub Actions and Cloud Build, with compatibility for Cloud Run deployment, ensuring scalability and ease of use. Currently, inference is performed using CPU. For setting up Cloud Run with Terraform, refer to the following repository for instructions:\n\n[FastAPI-CloudRun-Starter](https://github.com/mazzasaverio/fastapi-cloudrun-starter)\n\n## Getting Started\n\nTo get started with YouTube Auto-Dub, follow these steps:\n\n### 1. Environment Setup\n\nFor local development, we recommend setting up a conda environment with:\n\n```bash\nconda install mamba -n base -c conda-forge\nmamba create -n youtube-auto-dub python=3.9 -y\nmamba install -n youtube-auto-dub pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia -y\nconda activate youtube-auto-dub\npip install -r requirements.txt\n```\n\n### 2. Download Required Checkpoints\n\nDownload the model checkpoints necessary for voice timbre recognition and synthesis:\n\n```bash\nsudo aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://myshell-public-repo-hosting.s3.amazonaws.com/checkpoints_1226.zip -d /code -o checkpoints_1226.zip\nsudo unzip /code/checkpoints_1226.zip -d backend/checkpoints\n```\n\n### 3. Running the Application\n\nWith the environment set up and checkpoints downloaded, navigate to the backend directory and start the application using:\n\n```bash\ncd backend\nuvicorn app.main:app --reload\n```\n\n## Usage\n\nTo use YouTube Auto-Dub, begin by submitting a YouTube link via the endpoint:\n\n```\n/api/v1/download/\n```\n\nThe application will process the video, recognize the voice timbre, translate the subtitles, synthesize the translated speech matching the original timbre, and then assemble the final video. The processed video will be saved in `backend/data/final_videos`. With the video ID returned in the output, you can check the processing status through the endpoint:\n\n```\n/api/v1/status/{video_id}\n```\n\nFinally, you can download the final video by using the endpoint:\n\n```\n/api/v1/download-video/{video_id}\n```\n\ninserting the video's ID.\n\n## Deployment\n\nThis project is designed with cloud deployment in mind. The provided `cloudbuild.yaml` and Terraform configurations facilitate deployment on Google Cloud Platform, specifically using Cloud Run for scalable, serverless application hosting.\n\n## Contributing\n\nContributions are welcome! Whether you're fixing a bug, adding new features, or improving the documentation, your help is appreciated. Please feel free to fork the repository and submit pull requests.\n\n## Reference and Inspiration\n\nThe development of YouTube Auto-Dub was inspired by the following repository:\n\n- [OpenVoice](https://github.com/myshell-ai/OpenVoice): Instant voice cloning technology by MyShell, utilized for voice timbre recognition and synthesis in this project.\n\n## Future Directions\n\n- **Model Improvements**: Explore and integrate better models for voice recognition and synthesis.\n- **Serverless GPU Support**: Investigate options for serverless GPU computing to accelerate processing.\n- **Frontend Interface**: Develop a user-friendly frontend for easier interaction with the application.\n- **Translation Synchronization**: Enhance the synchronization between translated text and video content for a seamless viewing experience.\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmazzasaverio%2Fyoutube-auto-dub","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmazzasaverio%2Fyoutube-auto-dub","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmazzasaverio%2Fyoutube-auto-dub/lists"}