{"id":13456606,"url":"https://github.com/RVC-Boss/GPT-SoVITS","last_synced_at":"2025-03-24T11:30:41.062Z","repository":{"id":217456587,"uuid":"743230067","full_name":"RVC-Boss/GPT-SoVITS","owner":"RVC-Boss","description":"1 min voice data can also be used to train a good TTS model! (few shot voice cloning)","archived":false,"fork":false,"pushed_at":"2024-10-02T14:47:07.000Z","size":10521,"stargazers_count":34067,"open_issues_count":618,"forks_count":3905,"subscribers_count":205,"default_branch":"main","last_synced_at":"2024-10-14T20:26:17.389Z","etag":null,"topics":["text-to-speech","tts","vits","voice-clone","voice-cloneai","voice-cloning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RVC-Boss.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-14T18:05:21.000Z","updated_at":"2024-10-14T18:41:29.000Z","dependencies_parsed_at":"2024-01-17T18:22:02.560Z","dependency_job_id":"bb9b973b-2e8f-434f-802a-eb905eb671c1","html_url":"https://github.com/RVC-Boss/GPT-SoVITS","commit_stats":null,"previous_names":["rvc-boss/gpt-sovits"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RVC-Boss%2FGPT-SoVITS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RVC-Boss%2FGPT-SoVITS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RVC-Boss%2FGPT-SoVITS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RVC-Boss%2FGPT-SoVITS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RVC-Boss","download_url":"https://codeload.github.com/RVC-Boss/GPT-SoVITS/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245260655,"owners_count":20586436,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["text-to-speech","tts","vits","voice-clone","voice-cloneai","voice-cloning"],"created_at":"2024-07-31T08:01:24.817Z","updated_at":"2025-03-24T11:30:41.055Z","avatar_url":"https://github.com/RVC-Boss.png","language":"Python","funding_links":[],"categories":["Python","热门 LLM 项目","\u003cspan id=\"speech\"\u003eSpeech\u003c/span\u003e","语音合成","Text-to-Speech (TTS)","Trending LLM Projects","Tools \u0026 Frameworks","语音生成 (TTS) 🗣️🎙️","6. Generative Media Tools","TTS (Text-to-Speech) | 文本转语音","Repos","Table of Contents","zero-shot TTS","AI应用"],"sub_categories":["\u003cspan id=\"tool\"\u003eLLM (LLM \u0026 Tool)\u003c/span\u003e","网络服务_其他","Open-Source Models \u0026 Libraries","Open-source projects","Open Source TTS Models | 开源 TTS 模型","Model Tools"],"readme":"\u003cdiv align=\"center\"\u003e\n\n\n\u003ch1\u003eGPT-SoVITS-WebUI\u003c/h1\u003e\nA Powerful Few-shot Voice Conversion and Text-to-Speech WebUI.\u003cbr\u003e\u003cbr\u003e\n\n[![madewithlove](https://img.shields.io/badge/made_with-%E2%9D%A4-red?style=for-the-badge\u0026labelColor=orange)](https://github.com/RVC-Boss/GPT-SoVITS)\n\n\u003ca href=\"https://trendshift.io/repositories/7033\" target=\"_blank\"\u003e\u003cimg src=\"https://trendshift.io/api/badge/repositories/7033\" alt=\"RVC-Boss%2FGPT-SoVITS | Trendshift\" style=\"width: 250px; height: 55px;\" width=\"250\" height=\"55\"/\u003e\u003c/a\u003e\n\n\u003c!-- img src=\"https://counter.seku.su/cmoe?name=gptsovits\u0026theme=r34\" /\u003e\u003cbr\u003e --\u003e\n\n[![Open In Colab](https://img.shields.io/badge/Colab-F9AB00?style=for-the-badge\u0026logo=googlecolab\u0026color=525252)](https://colab.research.google.com/github/RVC-Boss/GPT-SoVITS/blob/main/colab_webui.ipynb)\n[![License](https://img.shields.io/badge/LICENSE-MIT-green.svg?style=for-the-badge)](https://github.com/RVC-Boss/GPT-SoVITS/blob/main/LICENSE)\n[![Huggingface](https://img.shields.io/badge/🤗%20-online%20demo-yellow.svg?style=for-the-badge)](https://huggingface.co/spaces/lj1995/GPT-SoVITS-v2)\n[![Discord](https://img.shields.io/discord/1198701940511617164?color=%23738ADB\u0026label=Discord\u0026style=for-the-badge)](https://discord.gg/dnrgs5GHfG)\n\n**English** | [**中文简体**](./docs/cn/README.md) | [**日本語**](./docs/ja/README.md) | [**한국어**](./docs/ko/README.md) | [**Türkçe**](./docs/tr/README.md)\n\n\u003c/div\u003e\n\n---\n\n## Features:\n\n1. **Zero-shot TTS:** Input a 5-second vocal sample and experience instant text-to-speech conversion.\n\n2. **Few-shot TTS:** Fine-tune the model with just 1 minute of training data for improved voice similarity and realism.\n\n3. **Cross-lingual Support:** Inference in languages different from the training dataset, currently supporting English, Japanese, Korean, Cantonese and Chinese.\n\n4. **WebUI Tools:** Integrated tools include voice accompaniment separation, automatic training set segmentation, Chinese ASR, and text labeling, assisting beginners in creating training datasets and GPT/SoVITS models.\n\n**Check out our [demo video](https://www.bilibili.com/video/BV12g4y1m7Uw) here!**\n\nUnseen speakers few-shot fine-tuning demo:\n\nhttps://github.com/RVC-Boss/GPT-SoVITS/assets/129054828/05bee1fa-bdd8-4d85-9350-80c060ab47fb\n\n**User guide: [简体中文](https://www.yuque.com/baicaigongchang1145haoyuangong/ib3g1e) | [English](https://rentry.co/GPT-SoVITS-guide#/)**\n\n## Installation\n\nFor users in China, you can [click here](https://www.codewithgpu.com/i/RVC-Boss/GPT-SoVITS/GPT-SoVITS-Official) to use AutoDL Cloud Docker to experience the full functionality online.\n\n### Tested Environments\n\n- Python 3.9, PyTorch 2.0.1, CUDA 11\n- Python 3.10.13, PyTorch 2.1.2, CUDA 12.3\n- Python 3.9, PyTorch 2.2.2, macOS 14.4.1 (Apple silicon)\n- Python 3.9, PyTorch 2.2.2, CPU devices\n\n_Note: numba==0.56.4 requires py\u003c3.11_\n\n### Windows\n\nIf you are a Windows user (tested with win\u003e=10), you can [download the integrated package](https://huggingface.co/lj1995/GPT-SoVITS-windows-package/resolve/main/GPT-SoVITS-beta.7z?download=true) and double-click on _go-webui.bat_ to start GPT-SoVITS-WebUI.\n\n**Users in China can [download the package here](https://www.yuque.com/baicaigongchang1145haoyuangong/ib3g1e/dkxgpiy9zb96hob4#KTvnO).**\n\n### Linux\n\n```bash\nconda create -n GPTSoVits python=3.9\nconda activate GPTSoVits\nbash install.sh\n```\n\n### macOS\n\n**Note: The models trained with GPUs on Macs result in significantly lower quality compared to those trained on other devices, so we are temporarily using CPUs instead.**\n\n1. Install Xcode command-line tools by running `xcode-select --install`.\n2. Install FFmpeg by running `brew install ffmpeg`.\n3. Install the program by running the following commands:\n\n```bash\nconda create -n GPTSoVits python=3.9\nconda activate GPTSoVits\npip install -r requirements.txt\n```\n\n### Install Manually\n\n#### Install FFmpeg\n\n##### Conda Users\n\n```bash\nconda install ffmpeg\n```\n\n##### Ubuntu/Debian Users\n\n```bash\nsudo apt install ffmpeg\nsudo apt install libsox-dev\nconda install -c conda-forge 'ffmpeg\u003c7'\n```\n\n##### Windows Users\n\nDownload and place [ffmpeg.exe](https://huggingface.co/lj1995/VoiceConversionWebUI/blob/main/ffmpeg.exe) and [ffprobe.exe](https://huggingface.co/lj1995/VoiceConversionWebUI/blob/main/ffprobe.exe) in the GPT-SoVITS root.\n\nInstall [Visual Studio 2017](https://aka.ms/vs/17/release/vc_redist.x86.exe) (Korean TTS Only)\n\n##### MacOS Users\n```bash\nbrew install ffmpeg\n```\n\n#### Install Dependences\n\n```bash\npip install -r requirements.txt\n```\n\n### Using Docker\n\n#### docker-compose.yaml configuration\n\n0. Regarding image tags: Due to rapid updates in the codebase and the slow process of packaging and testing images, please check [Docker Hub](https://hub.docker.com/r/breakstring/gpt-sovits) for the currently packaged latest images and select as per your situation, or alternatively, build locally using a Dockerfile according to your own needs.\n1. Environment Variables：\n   - is_half: Controls half-precision/double-precision. This is typically the cause if the content under the directories 4-cnhubert/5-wav32k is not generated correctly during the \"SSL extracting\" step. Adjust to True or False based on your actual situation.\n2. Volumes Configuration，The application's root directory inside the container is set to /workspace. The default docker-compose.yaml lists some practical examples for uploading/downloading content.\n3. shm_size： The default available memory for Docker Desktop on Windows is too small, which can cause abnormal operations. Adjust according to your own situation.\n4. Under the deploy section, GPU-related settings should be adjusted cautiously according to your system and actual circumstances.\n\n#### Running with docker compose\n\n```\ndocker compose -f \"docker-compose.yaml\" up -d\n```\n\n#### Running with docker command\n\nAs above, modify the corresponding parameters based on your actual situation, then run the following command:\n\n```\ndocker run --rm -it --gpus=all --env=is_half=False --volume=G:\\GPT-SoVITS-DockerTest\\output:/workspace/output --volume=G:\\GPT-SoVITS-DockerTest\\logs:/workspace/logs --volume=G:\\GPT-SoVITS-DockerTest\\SoVITS_weights:/workspace/SoVITS_weights --workdir=/workspace -p 9880:9880 -p 9871:9871 -p 9872:9872 -p 9873:9873 -p 9874:9874 --shm-size=\"16G\" -d breakstring/gpt-sovits:xxxxx\n```\n\n## Pretrained Models\n\n**Users in China can [download all these models here](https://www.yuque.com/baicaigongchang1145haoyuangong/ib3g1e/dkxgpiy9zb96hob4#nVNhX).**\n\n1. Download pretrained models from [GPT-SoVITS Models](https://huggingface.co/lj1995/GPT-SoVITS) and place them in `GPT_SoVITS/pretrained_models`.\n\n2. Download G2PW models from [G2PWModel_1.1.zip](https://paddlespeech.bj.bcebos.com/Parakeet/released_models/g2p/G2PWModel_1.1.zip), unzip and rename to `G2PWModel`, and then place them in `GPT_SoVITS/text`.(Chinese TTS Only)\n\n3. For UVR5 (Vocals/Accompaniment Separation \u0026 Reverberation Removal, additionally), download models from [UVR5 Weights](https://huggingface.co/lj1995/VoiceConversionWebUI/tree/main/uvr5_weights) and place them in `tools/uvr5/uvr5_weights`.\n\n    - If you want to use `bs_roformer` or `mel_band_roformer` models for UVR5, you can manually download the model and corresponding configuration file, and put them in `tools/uvr5/uvr5_weights`. **Rename the model file and configuration file, ensure that the model and configuration files have the same and corresponding names except for the suffix**. In addition, the model and configuration file names **must include `roformer`** in order to be recognized as models of the roformer class.\n\n    - The suggestion is to **directly specify the model type** in the model name and configuration file name, such as `mel_mand_roformer`, `bs_roformer`. If not specified, the features will be compared from the configuration file to determine which type of model it is. For example, the model `bs_roformer_ep_368_sdr_12.9628.ckpt` and its corresponding configuration file `bs_roformer_ep_368_sdr_12.9628.yaml` are a pair, `kim_mel_band_roformer.ckpt` and `kim_mel_band_roformer.yaml` are also a pair.\n\n4. For Chinese ASR (additionally), download models from [Damo ASR Model](https://modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/files), [Damo VAD Model](https://modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-pytorch/files), and [Damo Punc Model](https://modelscope.cn/models/damo/punc_ct-transformer_zh-cn-common-vocab272727-pytorch/files) and place them in `tools/asr/models`.\n\n5. For English or Japanese ASR (additionally), download models from [Faster Whisper Large V3](https://huggingface.co/Systran/faster-whisper-large-v3) and place them in `tools/asr/models`. Also, [other models](https://huggingface.co/Systran) may have the similar effect with smaller disk footprint.\n\n## Dataset Format\n\nThe TTS annotation .list file format:\n\n```\nvocal_path|speaker_name|language|text\n```\n\nLanguage dictionary:\n\n- 'zh': Chinese\n- 'ja': Japanese\n- 'en': English\n- 'ko': Korean\n- 'yue': Cantonese\n\nExample:\n\n```\nD:\\GPT-SoVITS\\xxx/xxx.wav|xxx|en|I like playing Genshin.\n```\n\n## Finetune and inference\n\n### Open WebUI\n\n#### Integrated Package Users\n\nDouble-click `go-webui.bat`or use `go-webui.ps1`\nif you want to switch to V1,then double-click`go-webui-v1.bat` or use `go-webui-v1.ps1`\n\n#### Others\n\n```bash\npython webui.py \u003clanguage(optional)\u003e\n```\n\nif you want to switch to V1,then\n\n```bash\npython webui.py v1 \u003clanguage(optional)\u003e\n```\nOr maunally switch version in WebUI\n\n### Finetune\n\n#### Path Auto-filling is now supported\n\n    1. Fill in the audio path\n    2. Slice the audio into small chunks\n    3. Denoise(optinal)\n    4. ASR\n    5. Proofreading ASR transcriptions\n    6. Go to the next Tab, then finetune the model\n\n### Open Inference WebUI\n\n#### Integrated Package Users\n\nDouble-click `go-webui-v2.bat` or use `go-webui-v2.ps1` ,then open the inference webui at  `1-GPT-SoVITS-TTS/1C-inference`\n\n#### Others\n\n```bash\npython GPT_SoVITS/inference_webui.py \u003clanguage(optional)\u003e\n```\nOR\n\n```bash\npython webui.py\n```\nthen open the inference webui at `1-GPT-SoVITS-TTS/1C-inference`\n\n## V2 Release Notes\n\nNew Features:\n\n1. Support Korean and Cantonese\n\n2. An optimized text frontend\n\n3. Pre-trained model extended from 2k hours to 5k hours\n\n4. Improved synthesis quality for low-quality reference audio\n\n    [more details](https://github.com/RVC-Boss/GPT-SoVITS/wiki/GPT%E2%80%90SoVITS%E2%80%90v2%E2%80%90features-(%E6%96%B0%E7%89%B9%E6%80%A7))\n\nUse v2 from v1 environment:\n\n1. `pip install -r requirements.txt` to update some packages\n\n2. Clone the latest codes from github.\n\n3. Download v2 pretrained models from [huggingface](https://huggingface.co/lj1995/GPT-SoVITS/tree/main/gsv-v2final-pretrained) and put them into `GPT_SoVITS\\pretrained_models\\gsv-v2final-pretrained`.\n\n    Chinese v2 additional: [G2PWModel_1.1.zip](https://paddlespeech.bj.bcebos.com/Parakeet/released_models/g2p/G2PWModel_1.1.zip)（Download G2PW models,  unzip and rename to `G2PWModel`, and then place them in `GPT_SoVITS/text`.\n\n## V3 Release Notes\n\nNew Features:\n\n1. The timbre similarity is higher, requiring less training data to approximate the target speaker (the timbre similarity is significantly improved using the base model directly without fine-tuning).\n\n2. GPT model is more stable, with fewer repetitions and omissions, and it is easier to generate speech with richer emotional expression.\n\n    [more details](https://github.com/RVC-Boss/GPT-SoVITS/wiki/GPT%E2%80%90SoVITS%E2%80%90v3%E2%80%90features-(%E6%96%B0%E7%89%B9%E6%80%A7))\n\nUse v3 from v2 environment:\n\n1. `pip install -r requirements.txt` to update some packages\n\n2. Clone the latest codes from github.\n\n3. Download v3 pretrained models (s1v3.ckpt, s2Gv3.pth and models--nvidia--bigvgan_v2_24khz_100band_256x folder) from [huggingface](https://huggingface.co/lj1995/GPT-SoVITS/tree/main) and put them into `GPT_SoVITS\\pretrained_models`.\n\n    additional: for Audio Super Resolution model, you can read [how to download](./tools/AP_BWE_main/24kto48k/readme.txt)\n\n\n## Todo List\n\n- [x] **High Priority:**\n\n  - [x] Localization in Japanese and English.\n  - [x] User guide.\n  - [x] Japanese and English dataset fine tune training.\n\n- [ ] **Features:**\n  - [x] Zero-shot voice conversion (5s) / few-shot voice conversion (1min).\n  - [x] TTS speaking speed control.\n  - [ ] ~~Enhanced TTS emotion control.~~ Maybe use pretrained finetuned preset GPT models for better emotion.\n  - [ ] Experiment with changing SoVITS token inputs to probability distribution of GPT vocabs (transformer latent).\n  - [x] Improve English and Japanese text frontend.\n  - [ ] Develop tiny and larger-sized TTS models.\n  - [x] Colab scripts.\n  - [x] Try expand training dataset (2k hours -\u003e 10k hours).\n  - [x] better sovits base model (enhanced audio quality)\n  - [ ] model mix\n\n## (Additional) Method for running from the command line\nUse the command line to open the WebUI for UVR5\n```\npython tools/uvr5/webui.py \"\u003cinfer_device\u003e\" \u003cis_half\u003e \u003cwebui_port_uvr5\u003e\n```\n\u003c!-- If you can't open a browser, follow the format below for UVR processing,This is using mdxnet for audio processing\n```\npython mdxnet.py --model --input_root --output_vocal --output_ins --agg_level --format --device --is_half_precision\n``` --\u003e\nThis is how the audio segmentation of the dataset is done using the command line\n```\npython audio_slicer.py \\\n    --input_path \"\u003cpath_to_original_audio_file_or_directory\u003e\" \\\n    --output_root \"\u003cdirectory_where_subdivided_audio_clips_will_be_saved\u003e\" \\\n    --threshold \u003cvolume_threshold\u003e \\\n    --min_length \u003cminimum_duration_of_each_subclip\u003e \\\n    --min_interval \u003cshortest_time_gap_between_adjacent_subclips\u003e\n    --hop_size \u003cstep_size_for_computing_volume_curve\u003e\n```\nThis is how dataset ASR processing is done using the command line(Only Chinese)\n```\npython tools/asr/funasr_asr.py -i \u003cinput\u003e -o \u003coutput\u003e\n```\nASR processing is performed through Faster_Whisper(ASR marking except Chinese)\n\n(No progress bars, GPU performance may cause time delays)\n```\npython ./tools/asr/fasterwhisper_asr.py -i \u003cinput\u003e -o \u003coutput\u003e -l \u003clanguage\u003e -p \u003cprecision\u003e\n```\nA custom list save path is enabled\n\n## Credits\n\nSpecial thanks to the following projects and contributors:\n\n### Theoretical Research\n- [ar-vits](https://github.com/innnky/ar-vits)\n- [SoundStorm](https://github.com/yangdongchao/SoundStorm/tree/master/soundstorm/s1/AR)\n- [vits](https://github.com/jaywalnut310/vits)\n- [TransferTTS](https://github.com/hcy71o/TransferTTS/blob/master/models.py#L556)\n- [contentvec](https://github.com/auspicious3000/contentvec/)\n- [hifi-gan](https://github.com/jik876/hifi-gan)\n- [fish-speech](https://github.com/fishaudio/fish-speech/blob/main/tools/llama/generate.py#L41)\n- [f5-TTS](https://github.com/SWivid/F5-TTS/blob/main/src/f5_tts/model/backbones/dit.py)\n- [shortcut flow matching](https://github.com/kvfrans/shortcut-models/blob/main/targets_shortcut.py)\n### Pretrained Models\n- [Chinese Speech Pretrain](https://github.com/TencentGameMate/chinese_speech_pretrain)\n- [Chinese-Roberta-WWM-Ext-Large](https://huggingface.co/hfl/chinese-roberta-wwm-ext-large)\n- [BigVGAN](https://github.com/NVIDIA/BigVGAN)\n### Text Frontend for Inference\n- [paddlespeech zh_normalization](https://github.com/PaddlePaddle/PaddleSpeech/tree/develop/paddlespeech/t2s/frontend/zh_normalization)\n- [split-lang](https://github.com/DoodleBears/split-lang)\n- [g2pW](https://github.com/GitYCC/g2pW)\n- [pypinyin-g2pW](https://github.com/mozillazg/pypinyin-g2pW)\n- [paddlespeech g2pw](https://github.com/PaddlePaddle/PaddleSpeech/tree/develop/paddlespeech/t2s/frontend/g2pw)\n### WebUI Tools\n- [ultimatevocalremovergui](https://github.com/Anjok07/ultimatevocalremovergui)\n- [audio-slicer](https://github.com/openvpi/audio-slicer)\n- [SubFix](https://github.com/cronrpc/SubFix)\n- [FFmpeg](https://github.com/FFmpeg/FFmpeg)\n- [gradio](https://github.com/gradio-app/gradio)\n- [faster-whisper](https://github.com/SYSTRAN/faster-whisper)\n- [FunASR](https://github.com/alibaba-damo-academy/FunASR)\n- [AP-BWE](https://github.com/yxlu-0102/AP-BWE)\n\nThankful to @Naozumi520 for providing the Cantonese training set and for the guidance on Cantonese-related knowledge.\n\n## Thanks to all contributors for their efforts\n\n\u003ca href=\"https://github.com/RVC-Boss/GPT-SoVITS/graphs/contributors\" target=\"_blank\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=RVC-Boss/GPT-SoVITS\" /\u003e\n\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRVC-Boss%2FGPT-SoVITS","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FRVC-Boss%2FGPT-SoVITS","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRVC-Boss%2FGPT-SoVITS/lists"}