{"id":28483798,"url":"https://github.com/voine/bert-vits2-mnn","last_synced_at":"2025-07-30T12:04:42.959Z","repository":{"id":289697503,"uuid":"971282182","full_name":"Voine/Bert-VITS2-MNN","owner":"Voine","description":"TTS System Bert-VITS2 Android Ver,  powered by  alibaba-MNN engine.","archived":false,"fork":false,"pushed_at":"2025-06-04T09:31:44.000Z","size":17700,"stargazers_count":61,"open_issues_count":2,"forks_count":4,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-04T16:56:05.294Z","etag":null,"topics":["android","android-app","bert","bert-vits2","cppjieba","mnn","tokenizer","tts","tts-android","tts-engines","vits"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Voine.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-23T09:32:52.000Z","updated_at":"2025-06-04T10:26:57.000Z","dependencies_parsed_at":"2025-04-24T16:25:28.664Z","dependency_job_id":"6e81c60b-a97a-422b-9488-54c5528563e7","html_url":"https://github.com/Voine/Bert-VITS2-MNN","commit_stats":null,"previous_names":["voine/bert-vits2-mnn"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/Voine/Bert-VITS2-MNN","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Voine%2FBert-VITS2-MNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Voine%2FBert-VITS2-MNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Voine%2FBert-VITS2-MNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Voine%2FBert-VITS2-MNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Voine","download_url":"https://codeload.github.com/Voine/Bert-VITS2-MNN/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Voine%2FBert-VITS2-MNN/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267865676,"owners_count":24157343,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-30T02:00:09.044Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["android","android-app","bert","bert-vits2","cppjieba","mnn","tokenizer","tts","tts-android","tts-engines","vits"],"created_at":"2025-06-07T22:05:45.394Z","updated_at":"2025-07-30T12:04:42.948Z","avatar_url":"https://github.com/Voine.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Bert-VITS2-MNN\n\n\u003e ✨ [Bert-VITS2](https://github.com/fishaudio/Bert-VITS2) Android 版, 推理框架基于 [alibaba-MNN](https://github.com/alibaba/MNN).\n\n---\n\n## 🧠 简介\n\n本工程提供了一个示例，实现了离线推理版本的 Bert-VITS2 （2.3版本），目前仅适配了中文：\n\n- 🧠 **蒸馏版中文 BERT 模型** ：中文 Bert 模型使用了一个自制的蒸馏版本，基于 [Wikipedia 中文](https://huggingface.co/datasets/pleisto/wikipedia-cn-20230720-filtered)以及 [SkyPile 中文数据集](https://huggingface.co/datasets/Skywork/SkyPile-150B)，共计约 1000W 条文本进行模型蒸馏，将体积缩减至 30M。（也不知道蒸的咋样反正最后看曲线是收敛了 -.-)\n- 🏗 **MNN** ：基于 MNN 推理框架实现 BV2 的整个推理流程，推理参考自其 onnx 推理代码。(pth 直接转不成功，你没资格啊，你没资格.jpg)\n- 🧹 **cppjieba** and **cpptokenizer** ：用来平替 Python 端的 jieba 分词以及 huggingface 的 tokenizer。一些 BV2 独有的文本预处理步骤使用 Kotlin 进行平替实现。(此过程 GPT 老祖帮了许多)\n\n整个过程在 Android 端全程 **离线推理** 无需任何联网服务.\n\n---\n\n## 🔬 大体流程\n\n```\nInput Text\n   ↓\nTokenization + G2P (cppjieba + tokenizer + kotlin code)\n   ↓\nBERT embedding (distilled Chinese model)\n   ↓\nEncoder + Emb + DP/SDP + Flow + Decoder (BV2 infer by MNN)\n   ↓\nWaveform output (.wav)\n```\n\n---\n\n## 🎵 示例音频\n\n此处提供一些中文音频示例，基于部分明日方舟语音集以及[原神语音集](https://www.bilibili.com/opus/804258696892776484)进行训练:\n\n| Text               | Character | Audio                                                                                      |\n|--------------------|-----------|--------------------------------------------------------------------------------------------|\n| 博士，当初在龙门，我不该放你走的。  | 陈         | 🔊 [Play](https://github.com/user-attachments/assets/a6fc4022-e473-41e3-89da-0f5c9741a4c4) |\n| 旅行者，好久不见。          | 珐露珊       | 🔊 [Play](https://github.com/user-attachments/assets/60a96546-1e18-43b8-9a6a-3c9bfd5eca42) |\n| 工作还没有做完，又要开始搬砖了。   | 甘雨        | 🔊 [Play](https://github.com/user-attachments/assets/7482e892-630f-47ee-829f-336ceb9525c4)                                                   |\n\n\n\n---\n\n## ⚡ 本地编译指南\n\n### Clone with submodules\n\n```bash\nGIT_LFS_SKIP_SMUDGE=1 git clone --recurse-submodules git@github.com:Voine/Bert-VITS2-MNN.git\n\n# for windows powershell\n$env:GIT_LFS_SKIP_SMUDGE=1; git clone --recurse-submodules git@github.com:Voine/Bert-VITS2-MNN.git\n\ncd Bert-VITS2-MNN\n```\n\nIf already cloned:\n\n```bash\ngit submodule update --init --recursive\n```\n\n### Build for Android\n\n\u003e 📦 建议使用 Android Studio 进行工程编译，用 IDE 打开根目录即可\n\n```bash\n# From project root\n./gradlew assembleRelease\n```\n\n---\n\n## 🛁 Git LFS \n\n本工程的一些文件如 `.mnn` ，使用 lfs 进行存储，需要按照如下方式拉代码：\n\n```bash\ngit lfs install\ngit lfs pull\n```\n\nTo track files (if contributing):\n\n```bash\ngit lfs track \"*.mnn\"\n```\n\n---\n\n## 🛠️ Submodule 依赖\n\n| Library      | Path                         |\n|--------------|------------------------------|\n| [MNN](https://github.com/alibaba/MNN)        | `third_party/MNN`            |\n| [cppjieba](https://github.com/yanyiwu/cppjieba)     | `third_party/cppjieba`       |\n| [tokenizer-cpp](https://github.com/mlc-ai/tokenizers-cpp) | `third_party/tokenizers-cpp` |\n\n\n---\n\n## 💡 关于 - 模型蒸馏 -\n\n中文模型基于 [chinese-roberta-wwm-ext-large](https://huggingface.co/hfl/chinese-roberta-wwm-ext-large) 进行蒸馏，为适配移动端，大幅缩减了体积。原版直接转换能有 1.2G ...\n\n蒸馏代码详见 [`distill/README.md`](distill/README.md).\n\n---\n\n\n## 💡 关于 - 自制模型替换 -\n\n1. 如果你需要替换自己的模型尝试验证，首先需要参考 [BertVITS2](https://github.com/fishaudio/Bert-VITS2) 内的说明进行训练得到桌面端模型，目前仅支持 2.3 版本，本工程基于的 BV2 代码 commit 为 13424595，如需自制模型，建议 BV2 代码版本保持一致。\n2. 将你的 pth 模型转换成 onnx, onnx 导出脚本在  [这里](https://github.com/fishaudio/Bert-VITS2/blob/master/export_onnx.py)\n3. 使用 [MNN Convert](https://mnn-docs.readthedocs.io/en/latest/tools/convert.html) 将所有模块的 onnx 模型转成 mnn\n4. 放到 assets/bv2_model 内，如果你的模型名字有变化，则需要修改 VoiceViewModel.kt 内关于模型路径加载的部分。（硬编码字符串一时爽，一直硬编码一直爽）\n\n---\n\n## 💡 关于 - third_party -\n\n目前在 third_party 内的 cppjieba、tokenizer-cpp 以及 MNN 仅是为了提供头文件，若需要自行编译 tokenizer-cpp 并替换产物 [libtokenizers_c.a](cpptokenizer/src/main/jniStaticLibs/arm64-v8a/libtokenizers_c.a) [libtokenizers_cpp.a](cpptokenizer/src/main/jniStaticLibs/arm64-v8a/libtokenizers_cpp.a)，需修改 [huggingface_tokenizer.cc](third_party/tokenizers-cpp/src/huggingface_tokenizer.cc) 内的 add_special_tokens 默认为 true\n\n---\n\n## 📋 工程大体结构\n\n```\n├── app/\n├──── src/main/                 \n│           ├── assets               # mnn bert model, cppjieba dic, mnn bv2model\n│           ├── java/preprocess      # Text preprocess code\n├── bertvits2                        # Bert-VITS2 infer code\n├── cppjieba                         # cppjieba interface \n├── cpptokenizer                     # cpptokenizer interface\n├── third_party                      # provide hpp\n\n```\n\n---\n\n## 🙌 鸣谢\n\n本工程基于以下前辈们的贡献做了一些微不足道的搬砖工作，也希望能为后续在端智能推理捣鼓的小伙伴提供一些参考。\n\n- [VITS](https://github.com/jaywalnut310/vits)\n- [BertVITS2](https://github.com/fishaudio/Bert-VITS2)\n- [MNN](https://github.com/alibaba/MNN)\n- [cppjieba](https://github.com/yanyiwu/cppjieba)\n- [tokenizer-cpp](https://github.com/mlc-ai/tokenizers-cpp)\n\n---\n\n## 🛠️ 后续工作\n\n- 看一下日文版和英文版怎么搞\n- 迁移到[移动版老婆聊天器](https://github.com/Voine/ChatWaifu_Mobile)中\n\n---\n\n## ✨  简介视频\n\n- [Video](https://www.bilibili.com/video/BV1f5Ldz5Enz)\n\n---\n\n\n## 免责声明\n### 本项目仅供学习交流使用，禁止用于商业用途，作者纯为爱发电搞着玩的。\n\n### 严禁将此项目用于一切违反《中华人民共和国宪法》，《中华人民共和国刑法》，《中华人民共和国治安管理处罚法》和《中华人民共和国民法典》之用途。\n### 严禁用于任何政治相关用途。\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoine%2Fbert-vits2-mnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvoine%2Fbert-vits2-mnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoine%2Fbert-vits2-mnn/lists"}