Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/CorentinJ/Real-Time-Voice-Cloning
Clone a voice in 5 seconds to generate arbitrary speech in real-time
https://github.com/CorentinJ/Real-Time-Voice-Cloning
deep-learning python pytorch tensorflow tts voice-cloning
Last synced: about 1 month ago
JSON representation
Clone a voice in 5 seconds to generate arbitrary speech in real-time
- Host: GitHub
- URL: https://github.com/CorentinJ/Real-Time-Voice-Cloning
- Owner: CorentinJ
- License: other
- Created: 2019-05-26T08:56:15.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2024-01-14T18:39:10.000Z (4 months ago)
- Last Synced: 2024-01-27T11:04:41.013Z (4 months ago)
- Topics: deep-learning, python, pytorch, tensorflow, tts, voice-cloning
- Language: Python
- Homepage:
- Size: 353 MB
- Stars: 49,712
- Watchers: 927
- Forks: 8,477
- Open Issues: 188
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Lists
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 36543 | (Python)
- awesome-stars-wjf0214 - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-diarization - Real-Time Voice Cloning - Time-Voice-Cloning?style=social) | d-vector | Python & PyTorch | Implementation of "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis" (SV2TTS) with a vocoder that works in real-time. | (Software / Speaker embedding)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - time (Python)
- awesome-github-repos - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- my-awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-list - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time. (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-deepfakes-materials - Real-Time Voice Cloning
- awesome-repositories - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome - 时时录音朗读 - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- Awesome-Machine-Learning - Real-Time Voice Cloning - stage deep learning framework that allows to create a numerical representation of a voice from a few seconds of audio, and to use it to condition a text-to-speech model trained to generalize to new voices. (Supplementary resources / 1.2 Books)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 36547 | (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 37402 | (Python)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 19529 | (Python)
- awesome-five23 - Real-Time-Voice-Cloning - time | CorentinJ | 21587 | (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- my-awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- my-awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 50999 | (Python)
- awesome-starts - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- docsify-awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 51006 | (Python)
- artsz-awesome - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 21699 | (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 49625 | (Python)
- awesome - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-from-stars - CorentinJ/Real-Time-Voice-Cloning - time (HarmonyOS / Windows Manager)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 51027 | (Python)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 51006 | (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - `★50930` Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-projects - Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-asv-antispoofing - Real-Time Voice Cloning - Time-Voice-Cloning?style=social) | d-vector | Python & PyTorch | Implementation of "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis" (SV2TTS) with a vocoder that works in real-time. | (Software / Speaker embedding)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 51027 | (Python)
- awesome-rainmana - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - `★49739` Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-hacking-lists - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-generative-models - real-time-voice-clone
- awesome-stars - Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-hacking-lists - Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python (1887))
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 51006 | (Python)
- awesome-hacking-lists - Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome - Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (Python)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 50931 | (Python)
- AiTreasureBox - CorentinJ/Real-Time-Voice-Cloning - 05-22_51043_17](https://img.shields.io/github/stars/CorentinJ/Real-Time-Voice-Cloning.svg)|Clone a voice in 5 seconds to generate arbitrary speech in real-time| (Repos)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 50993 | (Python)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 50993 | (Python)
- awesome-stars - Real-Time-Voice-Cloning - time | CorentinJ | 51027 | (Python)
- Awesome-GitHub-Repo - Real-Time-Voice-Cloning - 只需要你 5 秒钟的语音,就能生成你说出来的任何话,细思极恐。[<img src="https://tva1.sinaimg.cn/large/008i3skNly1gxlhtmg11mj305k05k746.jpg" alt="微信" width="18px" height="18px" />](https://mp.weixin.qq.com/s?__biz=MzUxNjg4NDEzNA%3D%3D&chksm=f9a22901ced5a017e5e32bb51b8650dd7063416119552a676c34be4a94302073c196666b4097&idx=1&mid=2247498440&scene=21&sn=19b48002eeeed367f70270d7652f9203#wechat_redirect) (好玩项目 / 黑科技)
- awesome-stars - CorentinJ/Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time (voice-cloning)
README
# Real-Time Voice Cloning
This repository is an implementation of [Transfer Learning from Speaker Verification to
Multispeaker Text-To-Speech Synthesis](https://arxiv.org/pdf/1806.04558.pdf) (SV2TTS) with a vocoder that works in real-time. This was my [master's thesis](https://matheo.uliege.be/handle/2268.2/6801).SV2TTS is a deep learning framework in three stages. In the first stage, one creates a digital representation of a voice from a few seconds of audio. In the second and third stages, this representation is used as reference to generate speech given arbitrary text.
**Video demonstration** (click the picture):
[![Toolbox demo](https://i.imgur.com/8lFUlgz.png)](https://www.youtube.com/watch?v=-O_hYhToKoA)
### Papers implemented
| URL | Designation | Title | Implementation source |
| --- | ----------- | ----- | --------------------- |
|[**1806.04558**](https://arxiv.org/pdf/1806.04558.pdf) | **SV2TTS** | **Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis** | This repo |
|[1802.08435](https://arxiv.org/pdf/1802.08435.pdf) | WaveRNN (vocoder) | Efficient Neural Audio Synthesis | [fatchord/WaveRNN](https://github.com/fatchord/WaveRNN) |
|[1703.10135](https://arxiv.org/pdf/1703.10135.pdf) | Tacotron (synthesizer) | Tacotron: Towards End-to-End Speech Synthesis | [fatchord/WaveRNN](https://github.com/fatchord/WaveRNN)
|[1710.10467](https://arxiv.org/pdf/1710.10467.pdf) | GE2E (encoder)| Generalized End-To-End Loss for Speaker Verification | This repo |## Heads up
Like everything else in Deep Learning, this repo is quickly getting old. Many other open-source repositories or SaaS apps (often paying) will give you a better audio quality than this repository will. If you care about the fidelity of the voice you're cloning, and its expressivity, here are some personal recommendations of alternative voice cloning solutions:
- Check out [CoquiTTS](https://github.com/coqui-ai/tts) for an open source repository that is more up-to-date, with a better voice cloning quality and more functionalities.
- Check out [paperswithcode](https://paperswithcode.com/task/speech-synthesis/) for other repositories and recent research in the field of speech synthesis.
- Check out [Resemble.ai](https://www.resemble.ai/) (disclaimer: I work there) for state of the art voice cloning with little hassle.## Setup
### 1. Install Requirements
1. Both Windows and Linux are supported. A GPU is recommended for training and for inference speed, but is not mandatory.
2. Python 3.7 is recommended. Python 3.5 or greater should work, but you'll probably have to tweak the dependencies' versions. I recommend setting up a virtual environment using `venv`, but this is optional.
3. Install [ffmpeg](https://ffmpeg.org/download.html#get-packages). This is necessary for reading audio files.
4. Install [PyTorch](https://pytorch.org/get-started/locally/). Pick the latest stable version, your operating system, your package manager (pip by default) and finally pick any of the proposed CUDA versions if you have a GPU, otherwise pick CPU. Run the given command.
5. Install the remaining requirements with `pip install -r requirements.txt`### 2. (Optional) Download Pretrained Models
Pretrained models are now downloaded automatically. If this doesn't work for you, you can manually download them [here](https://github.com/CorentinJ/Real-Time-Voice-Cloning/wiki/Pretrained-models).### 3. (Optional) Test Configuration
Before you download any dataset, you can begin by testing your configuration with:`python demo_cli.py`
If all tests pass, you're good to go.
### 4. (Optional) Download Datasets
For playing with the toolbox alone, I only recommend downloading [`LibriSpeech/train-clean-100`](https://www.openslr.org/resources/12/train-clean-100.tar.gz). Extract the contents as `/LibriSpeech/train-clean-100` where `` is a directory of your choosing. Other datasets are supported in the toolbox, see [here](https://github.com/CorentinJ/Real-Time-Voice-Cloning/wiki/Training#datasets). You're free not to download any dataset, but then you will need your own data as audio files or you will have to record it with the toolbox.### 5. Launch the Toolbox
You can then try the toolbox:`python demo_toolbox.py -d `
or
`python demo_toolbox.py`depending on whether you downloaded any datasets. If you are running an X-server or if you have the error `Aborted (core dumped)`, see [this issue](https://github.com/CorentinJ/Real-Time-Voice-Cloning/issues/11#issuecomment-504733590).