{"id":21069469,"url":"https://github.com/lissettecarlr/kuon","last_synced_at":"2025-05-16T04:34:26.414Z","repository":{"id":136183208,"uuid":"601873003","full_name":"lissettecarlr/kuon","owner":"lissettecarlr","description":"久远：一个开发中的大模型语音助手，当前关注易用性，简单上手，支持对话选择性记忆和Model Context Protocol (MCP)服务。 KUON:A large language model-based voice assistant under development, currently focused on ease of use and simple onboarding. It supports selective memory in conversations and the Model Context Protocol (MCP) service.","archived":false,"fork":false,"pushed_at":"2025-04-30T09:47:53.000Z","size":16019,"stargazers_count":45,"open_issues_count":0,"forks_count":11,"subscribers_count":1,"default_branch":"lite","last_synced_at":"2025-04-30T11:18:43.636Z","etag":null,"topics":["assistant","chatbot","mcp","openai","python","speech-to-text","text-to-speech","tts"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lissettecarlr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":"auditory/README.md","citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-02-15T02:12:55.000Z","updated_at":"2025-04-30T09:54:11.000Z","dependencies_parsed_at":"2023-10-27T08:29:58.687Z","dependency_job_id":"b8e350dc-6af0-4d5f-90fd-a18a5d0dfea9","html_url":"https://github.com/lissettecarlr/kuon","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lissettecarlr%2Fkuon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lissettecarlr%2Fkuon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lissettecarlr%2Fkuon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lissettecarlr%2Fkuon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lissettecarlr","download_url":"https://codeload.github.com/lissettecarlr/kuon/tar.gz/refs/heads/lite","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254470303,"owners_count":22076566,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["assistant","chatbot","mcp","openai","python","speech-to-text","text-to-speech","tts"],"created_at":"2024-11-19T18:35:39.420Z","updated_at":"2025-05-16T04:34:21.397Z","avatar_url":"https://github.com/lissettecarlr.png","language":"Python","funding_links":[],"categories":["📚 Projects (1974 total)"],"sub_categories":["MCP Servers"],"readme":"[首页](./README.md) | [旧版QQ-bot](https://github.com/lissettecarlr/kuon/tree/main) | [English](./README_en.md)\n\u003cp align=\"center\"\u003e\n \u003cimg src=\"./pic/logo.png\" align=\"middle\" width = \"300\"/\u003e\n\u003cp align=\"center\"\u003e\n\u003c/p\u003e\n\n# KUON\n\n久远，一个开发中的大模型语音助手。支持语音或文本输入，输出语音或文本。能够对接openai接口的对话模型。各个功能模块独立，可以自由组合。\n\n## 1 开发简述\n\n基于之前的分支又大刀阔斧的重构了一遍，主要将各个功能模块分离，使用接口的方式对接，而不是之前的融为一体，极大的提升了速度。也为了能之后在各个终端跑起来。\n\n最近从动画中提取了久远的语音，之后会训练语音模型。\n\n## 2 功能\n\n- [x] 使用文本输入交流，输出文本和语音\n- [x] 使用语音交流，输出文本和语音\n- [x] 检查功能，用于测试各部分是否正常\n- [x] 久远提示词\n- [x] 训练更好的vits模型\n- [x] 文本命令\n- [ ] 语音命令\n\n\n## 3 使用\n\n### 3.1 环境\n\n由于以将各个功能尽量剥离，所以配置也由原来的一个变成了多个，不过也就第一次改改，其实也还好。\n\n#### 3.1.1 基础环境\n\n* conda虚拟环境\n    ```bash\n    conda create -n kuon python=3.10\n    conda activate kuon\n    ```\n\n* pytorch（如果均使用接口方式则无需下列环境）:\n    ```bash\n    #  CUDA 11.8\n    pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118\n    ```\n\n* 安装基本软件包:\n    ```bash\n    pip install -r requirements.txt\n    ```\n    语言播放使用playsound，如果在windows上需要修改源码，在`Lib\\site-packages\\playsound.py`中移除utf-16进行解码的部分\n\n* 或者直接用conda配置文件克隆（替代上面几步）\n    导出：\n    ```bash\n    conda create --name kuon --file environment.yml\n    ```\n    导出\n    ```bash\n    conda env export \u003e environment.yml\n    ```\n\n#### 3.1.2 语言转文本功能\n\n现在被独立到仓库[AutomaticSpeechRecognition](https://github.com/lissettecarlr/AutomaticSpeechRecognition)。本仓库已经被添加相关代码到`kuonasr`文件夹，默认使用funasr接口方式，服务部署参考AutomaticSpeechRecognition仓库的[说明](https://github.com/lissettecarlr/AutomaticSpeechRecognition/blob/main/README.md)，需要使用其他方式自行参考该说明。\n\n使用前请根据funasr服务位置修改配置文件`kuonasr/config.yaml`\n```yaml\nfunasr:\n   url: ws://172.0.0.1:1234\n```\n\n可以使用下列脚本拉去最新代码（通常不需要）\n```bash\ncd script\npython asr_update.py\n```\n\n#### 3.1.3 文本转语言功能\n\n现在被独立到仓库[TextToSpeech](https://github.com/lissettecarlr/TextToSpeech)，仓库分为训练模型的fineturn和推理的kuontts，本仓库已经将推理代码添加到`kuontts`文件夹\n\n默认使用了接口方式，也即这里无需安装环境，也无需放入模型，需要根据部署的服务修改`kuontts/config.yaml`里的请求url和speaker。服务部署和其他问题请移步[TextToSpeech](https://github.com/lissettecarlr/TextToSpeech)仓库说明。\n\n离线方式使用则修改配置文件`kuontts/config.yaml`，将`online`改为`offline`，并将模型放入`kuontts/offline/OUTPUT_MODEL`中，我这里暂时训练了paimon的模型来用。\n\n如需要更新代码可以执行：\n```bash\ncd script\npython tts_update.py\n```\n\n\n#### 3.1.4 对话模型\n\n使用openai的api模式对接的模型服务，通过在llm中配置config.yaml指定对接对象。\n```yaml\nurl : http://172.0.0.1:1234/v1/chat/completions\nkey : qmdr-xxxx\nmodel : gpt-3.5-turbo-16k\ntimeout : 120 # 历史对话的遗忘时间\npreset : ./llm/kuon.json \n```\n\n其中`timeout`表示进过多次时间后情况历史对话。`preset`则是用于角色扮演的提示词地址。\n\n目前其他开源模型也有openai的api部署方式，自己部署后修改这个配置文件即可。这边我试着用qwen-7b，但是角色扮演的效果很差，目前还是gpt-4最优解。\n\n### 3.2 测试（可选）\n\n使用前可以测试测试各个功能是否正常，分为：语音输入、语音转文本、对话模型、文本转语音、播放五个部分。每个测试都是独立的，可以选择跳过。\n\n```bash\npython check.py\n```\n\n### 3.3 运行\n\n默认会打印所以日志，可以修改配置文件中的日志过滤器部分，例如只输出异常日志：\n```yaml\nlog_filter : True\nlog_filter_level : WARNING\n```\n\n在根目录的配置文件修改一些默认选项\n```yaml\n# 是否开启语音输出\nvoice_output_sw : True\n\n# 是否开启文本输出\ntext_output_sw : True\n\n# 是否启动时开启语音输入\naudio_input_sw : False\n```\n\n启动\n```bash\npython kuon.py\n```\n\n### 3.4 对话效果\n\n该效果主要受到模型和提示词的影响，通过在llm/config.yaml中配置。需要注意的是目前gpt提示词我是通过动画字幕进行填写，量会很大，当使用付费api时会急速消耗token哦。看之后针对效果进行精简。\n\n#### GPT4\n\n![gpt35](./pic/gpt4.png)\n\n#### gpt3.5\n\n![gpt35](./pic/gpt35.png)\n\n#### qwen-7b\n\n非常容易重复回答，不太适应目前大量的提示词，该模型感觉在特化了知识问答方向后其他能力变弱了。\n\n![qwen](./pic/qwen-chat.png)\n\n#### gpt-4o\n\n该模型效果好，价格便宜，应该是最适合的模型了\n\n![gpt4o](./pic/gpt-4o.png)\n\n\n## 4 其他\n\n####  4.1 无法输入声音\n\n在配置文件中可以调整麦克风的输入通道，设备上由那些通道可以通过命令查看，默认是1\n```\npython .\\utils\\get_input_channels.py\n输出\n麦克风 ID 0 - Microsoft 声音映射器 - Input\n麦克风 ID 1 - 麦克风 (WO Mic Device)\n```\n还能调节输入声音的阈值threshold，阈值越高，需要的声音越大才能触发输入。\n\n#### 4.2 代码通过black格式化\n\n[black仓库](https://github.com/psf/black)\n\n#### 4.3 语音转文本相关问题\n\n见仓库[AutomaticSpeechRecognition](https://github.com/lissettecarlr/AutomaticSpeechRecognition)，该仓库用于语音识别，目前有三种实现方式，分别是paraformer、whisper_online、funasr，能够独立使用\n\n#### 4.4 文本转语音相关问题\n\n见仓库[TextToSepeech](https://github.com/lissettecarlr/TextToSpeech),文本转语音，包含训练部分和推理部分，能本地运行也可服务器部署","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flissettecarlr%2Fkuon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flissettecarlr%2Fkuon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flissettecarlr%2Fkuon/lists"}