{"id":13456664,"url":"https://github.com/netease-youdao/EmotiVoice","last_synced_at":"2025-03-24T11:31:04.424Z","repository":{"id":206192875,"uuid":"716023793","full_name":"netease-youdao/EmotiVoice","owner":"netease-youdao","description":"EmotiVoice 😊: a Multi-Voice and Prompt-Controlled TTS Engine","archived":false,"fork":false,"pushed_at":"2024-08-13T10:23:52.000Z","size":3852,"stargazers_count":7341,"open_issues_count":113,"forks_count":626,"subscribers_count":63,"default_branch":"main","last_synced_at":"2024-10-16T14:41:41.134Z","etag":null,"topics":["ai","deep-learning","emotion","emotivoice","multi-speaker","prompt","python","pytorch","speech","speech-synthesis","style","text-to-speech","tts"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/netease-youdao.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-08T10:15:27.000Z","updated_at":"2024-10-16T14:19:35.000Z","dependencies_parsed_at":"2024-01-02T09:27:49.821Z","dependency_job_id":"4832bc02-bf53-4df4-9290-d073a62114a8","html_url":"https://github.com/netease-youdao/EmotiVoice","commit_stats":null,"previous_names":["netease-youdao/emotivoice"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netease-youdao%2FEmotiVoice","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netease-youdao%2FEmotiVoice/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netease-youdao%2FEmotiVoice/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netease-youdao%2FEmotiVoice/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/netease-youdao","download_url":"https://codeload.github.com/netease-youdao/EmotiVoice/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221962436,"owners_count":16908338,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","deep-learning","emotion","emotivoice","multi-speaker","prompt","python","pytorch","speech","speech-synthesis","style","text-to-speech","tts"],"created_at":"2024-07-31T08:01:25.707Z","updated_at":"2024-10-29T00:30:51.075Z","avatar_url":"https://github.com/netease-youdao.png","language":"Python","funding_links":[],"categories":["Python","精选文章","语音合成","Tools \u0026 Frameworks","Repos","\u003cspan id=\"speech\"\u003eSpeech\u003c/span\u003e"],"sub_categories":["文字转语音","网络服务_其他","Open-source projects","\u003cspan id=\"tool\"\u003eLLM (LLM \u0026 Tool)\u003c/span\u003e"],"readme":"\u003cdiv align=\"center\"\u003e\n\u003ca href=\"https://trendshift.io/repositories/4833\" target=\"_blank\"\u003e\u003cimg src=\"https://trendshift.io/api/badge/repositories/4833\" alt=\"netease-youdao%2FEmotiVoice | Trendshift\" style=\"width: 250px; height: 55px;\" width=\"250\" height=\"55\"/\u003e\u003c/a\u003e\n\n\u003cfont size=4\u003e README: EN | \u003ca href=\"./README.zh.md\"\u003e中文\u003c/a\u003e  \u003c/font\u003e\n    \u003ch1\u003eEmotiVoice 😊: a Multi-Voice and Prompt-Controlled TTS Engine\u003c/h1\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n    \u003ca href=\"./README.zh.md\"\u003e\u003cimg src=\"https://img.shields.io/badge/README-中文版本-red\"\u003e\u003c/a\u003e\n    \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003ca href=\"./LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-Apache--2.0-yellow\"\u003e\u003c/a\u003e\n    \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003ca href=\"https://twitter.com/YDopensource\"\u003e\u003cimg src=\"https://img.shields.io/badge/follow-%40YDOpenSource-1DA1F2?logo=twitter\u0026style={style}\"\u003e\u003c/a\u003e\n    \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n\u003c/div\u003e\n\u003cbr\u003e\n\n**EmotiVoice** is a powerful and modern open-source text-to-speech engine that is available to you at no cost. EmotiVoice speaks both English and Chinese, and with over 2000 different voices (refer to the [List of Voices](https://github.com/netease-youdao/EmotiVoice/wiki/😊-voice-wiki-page) for details). The most prominent feature is **emotional synthesis**, allowing you to create speech with a wide range of emotions, including happy, excited, sad, angry and others.\n\nAn easy-to-use web interface is provided. There is also a scripting interface for batch generation of results. \n\nHere are a few samples that EmotiVoice generates:\n\n\n- [Chinese audio sample](https://github.com/netease-youdao/EmotiVoice/assets/3909232/6426d7c1-d620-4bfc-ba03-cd7fc046a4fb)\n  \n- [English audio sample](https://github.com/netease-youdao/EmotiVoice/assets/3909232/8f272eba-49db-493b-b479-2d9e5a419e26)\n  \n- [Fun Chinese English audio sample](https://github.com/netease-youdao/EmotiVoice/assets/3909232/a0709012-c3ef-4182-bb0e-b7a2ba386f1c)\n\n## Demo\n\nA demo is hosted on Replicate, [EmotiVoice](https://replicate.com/bramhooimeijer/emotivoice).\n\n## Hot News\n\n- [x] Tuning voice speed is now supported in 'OpenAI-compatible-TTS API', thanks to [@john9405](https://github.com/john9405). [#90](https://github.com/netease-youdao/EmotiVoice/pull/90) [#67](https://github.com/netease-youdao/EmotiVoice/issues/67) [#77](https://github.com/netease-youdao/EmotiVoice/issues/77)\n\n- [x] [The EmotiVoice app for Mac](https://github.com/netease-youdao/EmotiVoice/releases/download/v0.3/emotivoice-1.0.0-arm64.dmg) was released on December 28th, 2023. Just download and taste EmotiVoice's offerings!\n\n- [x] [The EmotiVoice HTTP API](https://github.com/netease-youdao/EmotiVoice/wiki/HTTP-API) was released on December 6th, 2023. Easier to start, faster to use, and with **over 13,000 free calls**. Additionally, users can explore more captivating voices provided by [Zhiyun](https://ai.youdao.com/).\n- [x] [Voice Cloning with your personal data](https://github.com/netease-youdao/EmotiVoice/wiki/Voice-Cloning-with-your-personal-data) has been released on December 13th, 2023, along with [DataBaker Recipe](https://github.com/netease-youdao/EmotiVoice/tree/main/data/DataBaker) and [LJSpeech Recipe](https://github.com/netease-youdao/EmotiVoice/tree/main/data/LJspeech). \n\n## Features under development\n\n- [ ] Support for more languages, such as Japanese and Korean. [#19](https://github.com/netease-youdao/EmotiVoice/issues/19) [#22](https://github.com/netease-youdao/EmotiVoice/issues/22)\n\nEmotiVoice prioritizes community input and user requests. We welcome your feedback!\n\n## Quickstart\n\n### EmotiVoice Docker image\n\nThe easiest way to try EmotiVoice is by running the docker image. You need a machine with a NVidia GPU. If you have not done so, set up NVidia container toolkit by following the instructions for [Linux](https://www.server-world.info/en/note?os=Ubuntu_22.04\u0026p=nvidia\u0026f=2) or [Windows WSL2](https://github.com/nyp-sit/it3103/blob/main/nvidia-docker-wsl2.md). Then EmotiVoice can be run with,\n\n```sh\ndocker run -dp 127.0.0.1:8501:8501 syq163/emoti-voice:latest\n```\nThe Docker image was updated on January 4th, 2024. If you have an older version, please update it by running the following commands:\n```sh\ndocker pull syq163/emoti-voice:latest\ndocker run -dp 127.0.0.1:8501:8501 -p 127.0.0.1:8000:8000 syq163/emoti-voice:latest\n```\nNow open your browser and navigate to http://localhost:8501 to start using EmotiVoice's powerful TTS capabilities.\n\nStarting from this version, the 'OpenAI-compatible-TTS API' is now accessible via http://localhost:8000/.\n\n### Full installation\n\n```sh\nconda create -n EmotiVoice python=3.8 -y\nconda activate EmotiVoice\npip install torch torchaudio\npip install numpy numba scipy transformers soundfile yacs g2p_en jieba pypinyin pypinyin_dict\npython -m nltk.downloader \"averaged_perceptron_tagger_eng\"\n```\n\n### Prepare model files\n\nWe recommend that users refer to the wiki page [How to download the pretrained model files](https://github.com/netease-youdao/EmotiVoice/wiki/Pretrained-models) if they encounter any issues.\n\n```sh\ngit lfs install\ngit lfs clone https://huggingface.co/WangZeJun/simbert-base-chinese WangZeJun/simbert-base-chinese\n```\nor, you can run:\n```sh\ngit clone https://www.modelscope.cn/syq163/WangZeJun.git\n```\n\n### Inference\n\n1. You can download the [pretrained models](https://drive.google.com/drive/folders/1y6Xwj_GG9ulsAonca_unSGbJ4lxbNymM?usp=sharing) by simply running the following command:\n```sh\ngit clone https://www.modelscope.cn/syq163/outputs.git\n```\n2. The inference text format is `\u003cspeaker\u003e|\u003cstyle_prompt/emotion_prompt/content\u003e|\u003cphoneme\u003e|\u003ccontent\u003e`. \n\n  - inference text example: `8051|Happy|\u003csos/eos\u003e [IH0] [M] [AA1] [T] engsp4 [V] [OY1] [S] engsp4 [AH0] engsp1 [M] [AH1] [L] [T] [IY0] engsp4 [V] [OY1] [S] engsp1 [AE1] [N] [D] engsp1 [P] [R] [AA1] [M] [P] [T] engsp4 [K] [AH0] [N] [T] [R] [OW1] [L] [D] engsp1 [T] [IY1] engsp4 [T] [IY1] engsp4 [EH1] [S] engsp1 [EH1] [N] [JH] [AH0] [N] . \u003csos/eos\u003e|Emoti-Voice - a Multi-Voice and Prompt-Controlled T-T-S Engine`.\n4. You can get phonemes by `python frontend.py data/my_text.txt \u003e data/my_text_for_tts.txt`.\n\n5. Then run:\n```sh\nTEXT=data/inference/text\npython inference_am_vocoder_joint.py \\\n--logdir prompt_tts_open_source_joint \\\n--config_folder config/joint \\\n--checkpoint g_00140000 \\\n--test_file $TEXT\n```\nthe synthesized speech is under `outputs/prompt_tts_open_source_joint/test_audio`.\n\n1. Or if you just want to use the interactive TTS demo page, run:\n```sh\npip install streamlit\nstreamlit run demo_page.py\n```\n\n### OpenAI-compatible-TTS API\n\nThanks to @lewangdev for adding an OpenAI compatible API [#60](../../issues/60). To set it up, use the following command:\n\n```sh\npip install fastapi pydub uvicorn[standard] pyrubberband\nuvicorn openaiapi:app --reload\n```\n\n### Wiki page\n\nYou may find more information from our [wiki](https://github.com/netease-youdao/EmotiVoice/wiki) page.\n\n## Training\n\n[Voice Cloning with your personal data](https://github.com/netease-youdao/EmotiVoice/wiki/Voice-Cloning-with-your-personal-data) has been released on December 13th, 2023.\n\n\n## Roadmap \u0026 Future work\n\n- Our future plan can be found in the [ROADMAP](./ROADMAP.md) file.\n- The current implementation focuses on emotion/style control by prompts. It uses only pitch, speed, energy, and emotion as style factors, and does not use gender. But it is not complicated to change it to style/timbre control.\n- Suggestions are welcome. You can file issues or [@ydopensource](https://twitter.com/YDopensource) on twitter.\n\n\n## WeChat group\nWelcome to scan the QR code below and join the WeChat group.\n\n\u003cimg src=\"https://github.com/netease-youdao/EmotiVoice/assets/49354974/cc3f4c8b-8369-4e50-89cc-e40d27a6bdeb\" alt=\"qr\" width=\"150\"/\u003e\n\n## Credits\n\n- [PromptTTS](https://speechresearch.github.io/prompttts/). The PromptTTS paper is a key basis of this project.\n- [LibriTTS](https://www.openslr.org/60/). The LibriTTS dataset is used in training of EmotiVoice.\n- [HiFiTTS](https://www.openslr.org/109/). The HiFi TTS dataset is used in training of EmotiVoice.\n- [ESPnet](https://github.com/espnet/espnet). \n- [WeTTS](https://github.com/wenet-e2e/wetts)\n- [HiFi-GAN](https://github.com/jik876/hifi-gan)\n- [Transformers](https://github.com/huggingface/transformers)\n- [tacotron](https://github.com/keithito/tacotron)\n- [KAN-TTS](https://github.com/alibaba-damo-academy/KAN-TTS)\n- [StyleTTS](https://github.com/yl4579/StyleTTS)\n- [Simbert](https://github.com/ZhuiyiTechnology/simbert)\n- [cn2an](https://github.com/Ailln/cn2an). EmotiVoice incorporates cn2an for number processing.\n\n## License\n\nEmotiVoice is provided under the Apache-2.0 License - see the [LICENSE](./LICENSE) file for details.\n\nThe interactive page is provided under the [User Agreement](./EmotiVoice_UserAgreement_易魔声用户协议.pdf) file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetease-youdao%2FEmotiVoice","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetease-youdao%2FEmotiVoice","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetease-youdao%2FEmotiVoice/lists"}