{"id":13794056,"url":"https://github.com/rotten-work/vits-mandarin-windows","last_synced_at":"2025-05-12T20:31:29.137Z","repository":{"id":62924732,"uuid":"554846881","full_name":"rotten-work/vits-mandarin-windows","owner":"rotten-work","description":"VITS for Mandarin. Support Windows and Linux, low-end and high-end hardwares","archived":false,"fork":true,"pushed_at":"2023-05-06T10:29:48.000Z","size":5965,"stargazers_count":112,"open_issues_count":4,"forks_count":13,"subscribers_count":1,"default_branch":"mandarin","last_synced_at":"2024-11-18T08:55:59.447Z","etag":null,"topics":["chinese","linux","mandarin","pytorch","vits","windows-desktop"],"latest_commit_sha":null,"homepage":"https://colab.research.google.com/drive/1uFUnZDbHMqKWBUQDZKih56Vkj2ixTN9B#scrollTo=p16LO6nXVDzP","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"jaywalnut310/vits","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rotten-work.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-10-20T13:53:36.000Z","updated_at":"2024-09-23T21:25:22.000Z","dependencies_parsed_at":"2023-02-13T00:45:58.510Z","dependency_job_id":null,"html_url":"https://github.com/rotten-work/vits-mandarin-windows","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rotten-work%2Fvits-mandarin-windows","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rotten-work%2Fvits-mandarin-windows/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rotten-work%2Fvits-mandarin-windows/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rotten-work%2Fvits-mandarin-windows/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rotten-work","download_url":"https://codeload.github.com/rotten-work/vits-mandarin-windows/tar.gz/refs/heads/mandarin","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253816751,"owners_count":21968877,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chinese","linux","mandarin","pytorch","vits","windows-desktop"],"created_at":"2024-08-03T23:00:35.177Z","updated_at":"2025-05-12T20:31:24.116Z","avatar_url":"https://github.com/rotten-work.png","language":"Jupyter Notebook","funding_links":[],"categories":["Modified"],"sub_categories":["Other Languages"],"readme":"# VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech\n\n### Jaehyeon Kim, Jungil Kong, and Juhee Son\n\nIn our recent [paper](https://arxiv.org/abs/2106.06103), we propose VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech.\n\nSeveral recent end-to-end text-to-speech (TTS) models enabling single-stage training and parallel sampling have been proposed, but their sample quality does not match that of two-stage TTS systems. In this work, we present a parallel end-to-end TTS method that generates more natural sounding audio than current two-stage models. Our method adopts variational inference augmented with normalizing flows and an adversarial training process, which improves the expressive power of generative modeling. We also propose a stochastic duration predictor to synthesize speech with diverse rhythms from input text. With the uncertainty modeling over latent variables and the stochastic duration predictor, our method expresses the natural one-to-many relationship in which a text input can be spoken in multiple ways with different pitches and rhythms. A subjective human evaluation (mean opinion score, or MOS) on the LJ Speech, a single speaker dataset, shows that our method outperforms the best publicly available TTS systems and achieves a MOS comparable to ground truth.\n\nVisit our [demo](https://jaywalnut310.github.io/vits-demo/index.html) for audio samples.\n\nWe also provide the [pretrained models](https://drive.google.com/drive/folders/1ksarh-cJf3F5eKJjLVWY0X1j1qsQqiS2?usp=sharing).\n\n** Update note: Thanks to [Rishikesh (ऋषिकेश)](https://github.com/jaywalnut310/vits/issues/1), our interactive TTS demo is now available on [Colab Notebook](https://colab.research.google.com/drive/1CO61pZizDj7en71NQG_aqqKdGaA_SaBf?usp=sharing).\n\n\u003ctable style=\"width:100%\"\u003e\n  \u003ctr\u003e\n    \u003cth\u003eVITS at training\u003c/th\u003e\n    \u003cth\u003eVITS at inference\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"resources/fig_1a.png\" alt=\"VITS at training\" height=\"400\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"resources/fig_1b.png\" alt=\"VITS at inference\" height=\"400\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\n## Pre-requisites\n0. Python \u003e= 3.6\n0. Clone this repository\n0. Install python requirements. Please refer [requirements.txt](requirements.txt)\n    1. You may need to install espeak first: `apt-get install espeak`\n0. Download datasets\n    1. Download and extract the LJ Speech dataset, then rename or create a link to the dataset folder: `ln -s /path/to/LJSpeech-1.1/wavs DUMMY1`\n    1. For mult-speaker setting, download and extract the VCTK dataset, and downsample wav files to 22050 Hz. Then rename or create a link to the dataset folder: `ln -s /path/to/VCTK-Corpus/downsampled_wavs DUMMY2`\n0. Build Monotonic Alignment Search and run preprocessing if you use your own datasets.\n```sh\n# Cython-version Monotonoic Alignment Search\ncd monotonic_align\npython setup.py build_ext --inplace\n\n# Preprocessing (g2p) for your own datasets. Preprocessed phonemes for LJ Speech and VCTK have been already provided.\n# python preprocess.py --text_index 1 --filelists filelists/ljs_audio_text_train_filelist.txt filelists/ljs_audio_text_val_filelist.txt filelists/ljs_audio_text_test_filelist.txt \n# python preprocess.py --text_index 2 --filelists filelists/vctk_audio_sid_text_train_filelist.txt filelists/vctk_audio_sid_text_val_filelist.txt filelists/vctk_audio_sid_text_test_filelist.txt\n```\n\n\n## Training Exmaple\n```sh\n# LJ Speech\npython train.py -c configs/ljs_base.json -m ljs_base\n\n# VCTK\npython train_ms.py -c configs/vctk_base.json -m vctk_base\n```\n\n\n## Inference Example\nSee [inference.ipynb](inference.ipynb)\n\n\u003cbr\u003e\n\n## 补充说明\n### 项目特点\n- 支持Windows和Linux，两个平台上都可以进行训练和推断\n- 兼容最新版本的各个依赖库\n- Windows平台所需特殊环境配置和操作说明\n- 支持中文和英文\n- 本项目添加了一个简易的面向对象风格的[推断脚本](inference.py)。\n- [这里](https://colab.research.google.com/drive/1uFUnZDbHMqKWBUQDZKih56Vkj2ixTN9B)是一个简单的Colab notebook，展示了如何使用该项目进行训练和推断的步骤。\n- [这里](https://colab.research.google.com/drive/1VWBOp3PDGNO77_xOm20yRtc4CSmsbqtb)是一个简单的Colab notebook，展示了如何使用预训练权重进行迁移训练（精调）\n- 预处理好的几套音频数据集以方便大家学习实验\n\n\n### Windows平台环境配置\n#### 安装PyTorch的GPU版本\n在Windows平台，\u003ccode\u003epip install -r requirements.txt\u003c/code\u003e 安装的是CPU版本的PyTorch。所以需要去[PyTorch官网](https://pytorch.org)挑选并运行合适的GPU版本PyTorch安装命令。下面命令仅供参考：\n```\nconda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia\n```\n\n#### eSpeak的配置\n- 在Windows平台上用英文做训练或推断的话，需要安装[eSpeak Ng](https://github.com/espeak-ng/espeak-ng)库。[这里](https://github.com/espeak-ng/espeak-ng/releases)是下载页面，推荐使用.msi安装。\n- 安装eSpeak Ng后，请添加环境变量PHONEMIZER_ESPEAK_LIBRARY，并将变量值设置为{INSTALLDIR}\\libespeak-ng.dll。如图所示：\u003cbr\u003e\n\u003cimg src=\"resources/PHONEMIZER_ESPEAK_LIBRARY.png\"\u003e\n\n#### 构建Monotonoic Alignment Search扩展模块\n请先下载安装Visual Studio。到[这里](https://visualstudio.microsoft.com/#vs-section)下载。\n\n### 数据集\n\u003ctable style=\"width:100%\"\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e标贝中文标准女声音库（处理后）16-bit PCM WAV，22050 Hz\u003c/td\u003e\n    \u003ctd\u003e\n      链接：https://pan.baidu.com/s/1oihti9-aoJ447l54kdjChQ \u003cbr\u003e\n      提取码：vits \n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eLJSpeech数据集16-bit PCM WAV，22050 Hz\u003c/td\u003e\n    \u003ctd\u003e\n      链接：https://pan.baidu.com/s/1q2A38znFmxn3zCn587ZKkw \u003cbr\u003e\n      提取码：vits\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e标贝中文标准女声音库官网\u003c/td\u003e\n    \u003ctd\u003ehttps://www.data-baker.com/data/index/TNtts/\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eLJSpeech数据集官网\u003c/td\u003e\n    \u003ctd\u003ehttps://keithito.com/LJ-Speech-Dataset/\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\u003cbr\u003e\n\n### 预训练权重\n\u003ctable style = \"width:100%\"\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e标贝中文标准女声音库预训练权重\u003c/td\u003e\n    \u003ctd\u003e\n      链接：https://pan.baidu.com/s/1pN-wL_5wB9gYMAr2Mh7Jvg \u003cbr\u003e\n      提取码：vits\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n注：各预训练权重文件包括生成网络权重（G开头），鉴别器网络权重（D开头），还有训练时使用的cleaners与symbols（方便与其他VITS仓库的代码或工具兼容）\u003cbr\u003e\u003cbr\u003e\n\n## 效果展示\n### [Gallery](gallery/Gallery.md) \u003cbr\u003e\u003cbr\u003e\n\n## 参考与鸣谢\n### 大佬们的VITS语音合成GitHub仓库\n*   https://github.com/jaywalnut310/vits\n*   https://github.com/CjangCjengh/vits\n*   https://github.com/AlexandaJerry/vits-mandarin-biaobei\n*   https://github.com/JOETtheIV/VITS-Paimon\n*   https://github.com/w4123/vits\n*   https://github.com/xiaoyou-bilibili/tts_vits\n*   https://github.com/wind4000/vits.git\n### 参考B站链接\n*   【CV失业计划】基于VITS神经网络模型的近乎完美派蒙中文语音合成：\\\n  https://www.bilibili.com/video/BV1rB4y157fd\n*   【原神】派蒙Vtuber出道计划——基于AI深度学习VITS和VSeeFace的派蒙语音合成/套皮：\\\nhttps://www.bilibili.com/video/BV16G4y1B7Ey\n*   【深度学习】基于vits的语音合成：\\\nhttps://www.bilibili.com/video/BV1Fe4y1r737\n*   零基础炼丹 - vits版补充：\\\nhttps://www.bilibili.com/read/cv18357171\n\n\n## 恰饭\n生活不易，喵喵叹气。。。如果您喜欢该项目，请对该项目star一下表示支持吧~ \u003cbr\u003e \u003cimg src=\"resources/恰饭512.png\"\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frotten-work%2Fvits-mandarin-windows","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frotten-work%2Fvits-mandarin-windows","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frotten-work%2Fvits-mandarin-windows/lists"}