https://github.com/moeru-ai/airi
💖🧸 A container of souls of AI waifu / virtual characters to bring them into our worlds, wishing to achieve Neuro-sama's altitude, completely LLM and AI driven, capable of realtime voice chat, Minecraft playing, Factorio playing. Can be run in Browser or Desktop.
https://github.com/moeru-ai/airi
ai-vtuber chat-ui chatui live2d llm moeru-ai neuro-sama neurosama vrm vtuber
Last synced: 5 months ago
JSON representation
💖🧸 A container of souls of AI waifu / virtual characters to bring them into our worlds, wishing to achieve Neuro-sama's altitude, completely LLM and AI driven, capable of realtime voice chat, Minecraft playing, Factorio playing. Can be run in Browser or Desktop.
- Host: GitHub
- URL: https://github.com/moeru-ai/airi
- Owner: moeru-ai
- License: mit
- Created: 2024-12-01T16:33:36.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2025-05-11T09:06:18.000Z (5 months ago)
- Last Synced: 2025-05-12T04:02:34.326Z (5 months ago)
- Topics: ai-vtuber, chat-ui, chatui, live2d, llm, moeru-ai, neuro-sama, neurosama, vrm, vtuber
- Language: Vue
- Homepage: https://airi.moeru.ai/docs/
- Size: 88.7 MB
- Stars: 743
- Watchers: 5
- Forks: 57
- Open Issues: 10
-
Metadata Files:
- Readme: README.ja-JP.md
- Changelog: changelogithub.config.ts
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
- Security: .github/SECURITY.md
Awesome Lists containing this project
- awesome - moeru-ai/airi - 💖🧸 Self hosted, you owned Grok Companion, a container of souls of waifu, cyber livings to bring them into our worlds, wishing to achieve Neuro-sama's altitude. Capable of realtime voice chat, Minecraft, Factorio playing. Web / macOS / Windows supported. (Vue)
- starred - moeru-ai/airi - 💖🧸 Self hosted, you owned Grok Companion, a container of souls of waifu, cyber livings to bring them into our worlds, wishing to achieve Neuro-sama's altitude. Capable of realtime voice chat, Minecraf (Vue)
- StarryDivineSky - moeru-ai/airi - ai团队开发的开源项目，它以"自托管"的形式提供类似Grok Companion的功能，被描述为"waifu灵魂的容器"，旨在通过赛博技术将虚拟角色（waifu）的意识与现实世界连接，目标是实现接近Neuro-sama（神经元之神）级别的AI能力。该项目通过容器化技术实现虚拟角色的实时交互功能，支持Web、macOS和Windows多平台运行，核心功能包括实时语音聊天、Minecraft（我的世界）和Factorio（异星工厂）等游戏的操控能力。其技术原理基于容器化架构，将虚拟角色的"灵魂"（可能指AI模型或虚拟形象）封装在可独立运行的容器中，通过API接口与用户设备交互，实现跨平台的实时语音通信和游戏控制。项目特色在于强调用户数据的自主性（自托管模式），允许用户完全掌控虚拟角色的运行环境，同时通过容器技术降低部署门槛，使用户能够便捷地在不同操作系统上运行该系统。这种设计既满足了对虚拟角色互动体验的追求，也兼顾了隐私保护和本地化部署的需求。 (多模态大模型 / 资源传输下载)
- AiTreasureBox - moeru-ai/airi - 09-11_13497_79](https://img.shields.io/github/stars/moeru-ai/airi.svg)|💖🧸 Self hosted, you owned Grok Companion, a container of souls of waifu, cyber livings to bring them into our worlds, wishing to achieve Neuro-sama's altitude. Capable of realtime voice chat, Minecraft, Factorio playing. Web / macOS / Windows supported.| (Repos)
README
アイリ VTuber
> Heavily inspired by [Neuro-sama](https://www.youtube.com/@Neurosama)
Unlike the other AI driven VTuber open source projects, アイリ VTuber was built with many support of Web technologies such as [WebGPU](https://www.w3.org/TR/webgpu/), [WebAudio](https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API), [Web Workers](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers), [WebAssembly](https://webassembly.org/), [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket), etc. from the first day.
This means that **アイリ VTuber is capable to run on modern browsers and devices**, and even on mobile devices (already done with PWA support), this brought a lot of possibilities for us (the developers) to build and extend the power of アイリ VTuber to the next level, while still left the flexibilities for users to enable features that requires TCP connections or other non-Web technologies such as connect to voice channel to Discord, or playing Minecraft, Factorio with you and your friends.
> [!NOTE]
>
> We are still in the early stage of development where we are seeking out talented developers to join us and help us to make アイリ VTuber a reality.
>
> It's ok if you are not familiar with Vue.js, TypeScript, and devtools that required for this project, you can join us as an artist, designer, or even help us to launch our first live stream.
>
> Even you are a big fan of React or Svelte, even Solid, we welcome you, you can open a sub-directory to add features that you want to see in アイリ VTuber, or would like to experiment with.
>
> Fields (and related projects) that we are looking for:
>
> - Live2D modeller
> - VRM modeller
> - VRChat avatar designer
> - Computer Vision
> - Reinforcement Learning
> - Speech Recognition
> - Speech Synthesis
> - ONNX Runtime
> - Transformers.js
> - vLLM
> - WebGPU
> - Three.js
> - WebXR ([checkout the another project](https://github.com/moeru-ai/n3p6) we have under @moeru-ai organization)
>
> **If you are interested in, why not introduce yourself here? [Would like to join part of us to build Airi?](https://github.com/moeru-ai/airi/discussions/33)**
## Current progress
Capable of
- [x] Brain
- [x] Play [Minecraft](https://www.minecraft.net)
- [x] Play [Factorio](https://www.factorio.com) (WIP, but [PoC and demo available](https://github.com/moeru-ai/airi-factorio))
- [x] Chat in [Telegram](https://telegram.org)
- [x] Chat in [Discord](https://discord.com)
- [ ] Memory
- [x] Pure in-browser database support (DuckDB WASM | `pglite`)
- [ ] Memory Alaya (WIP)
- [ ] Pure in-browser local (WebGPU) inference
- [x] Ears
- [x] Audio input from browser
- [x] Audio input from [Discord](https://discord.com)
- [x] Client side speech recognition
- [x] Client side talking detection
- [x] Mouth
- [x] [ElevenLabs](https://elevenlabs.io/) voice synthesis
- [x] Body
- [x] VRM support
- [x] Control VRM model
- [x] VRM model animations
- [x] Auto blink
- [x] Auto look at
- [x] Idle eye movement
- [x] Live2D support
- [x] Control Live2D model
- [x] Live2D model animations
- [x] Auto blink
- [x] Auto look at
- [x] Idle eye movement
## Development
```shell
pnpm i
```
```shell
pnpm dev
```
## Supported the following LLM API Providers (powered by [xsai](https://github.com/moeru-ai/xsai))
- [x] [OpenRouter](https://openrouter.ai/)
- [x] [vLLM](https://github.com/vllm-project/vllm)
- [x] [SGLang](https://github.com/sgl-project/sglang)
- [x] [Ollama](https://github.com/ollama/ollama)
- [x] [Google Gemini](https://developers.generativeai.google)
- [x] [OpenAI](https://platform.openai.com/docs/guides/gpt/chat-completions-api)
- [ ] [Azure OpenAI API](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference) (PR welcome)
- [x] [Anthropic Claude](https://anthropic.com)
- [ ] [AWS Claude](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference) (PR welcome)
- [x] [DeepSeek](https://www.deepseek.com/)
- [x] [Qwen](https://help.aliyun.com/document_detail/2400395.html)
- [x] [xAI](https://x.ai/)
- [x] [Groq](https://wow.groq.com/)
- [x] [Mistral](https://mistral.ai/)
- [x] [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/)
- [x] [Together.ai](https://www.together.ai/)
- [x] [Fireworks.ai](https://www.together.ai/)
- [x] [Novita](https://www.novita.ai/)
- [x] [Zhipu](https://bigmodel.cn)
- [x] [SiliconFlow](https://cloud.siliconflow.cn/i/rKXmRobW)
- [x] [Stepfun](https://platform.stepfun.com/)
- [x] [Baichuan](https://platform.baichuan-ai.com)
- [x] [Minimax](https://api.minimax.chat/)
- [x] [Moonshot AI](https://platform.moonshot.cn/)
- [x] [Tencent Cloud](https://cloud.tencent.com/document/product/1729)
- [ ] [Sparks](https://www.xfyun.cn/doc/spark/Web.html) (PR welcome)
- [ ] [Volcano Engine](https://www.volcengine.com/experience/ark?utm_term=202502dsinvite&ac=DSASUQY5&rc=2QXCA1VI) (PR welcome)
## Sub-projects born from this project
- [`unspeech`](https://github.com/moeru-ai/unspeech): Universal endpoint proxy server for `/audio/transcriptions` and `/audio/speech`, like LiteLLM but for any ASR and TTS
- [`hfup`](https://github.com/moeru-ai/hfup): tools to help on deploying, bundling to HuggingFace Spaces
- [`@proj-airi/drizzle-duckdb-wasm`](https://github.com/moeru-ai/airi/tree/main/packages/drizzle-duckdb-wasm/README.md): Drizzle ORM driver for DuckDB WASM
- [`@proj-airi/duckdb-wasm`](https://github.com/moeru-ai/airi/tree/main/packages/duckdb-wasm/README.md): Easy to use wrapper for `@duckdb/duckdb-wasm`
- [`@proj-airi/lobe-icons`](https://github.com/proj-airi/lobe-icons): Iconify JSON bundle for amazing AI & LLM icons from [lobe-icons](https://github.com/lobehub/lobe-icons), support Tailwind and UnoCSS
- [Airi Factorio](https://github.com/moeru-ai/airi-factorio): Allow Airi to play Factorio
- [Factorio RCON API](https://github.com/nekomeowww/factorio-rcon-api): RESTful API wrapper for Factorio headless server console
- [`autorio`](https://github.com/moeru-ai/airi-factorio/tree/main/packages/autorio): Factorio automation library
- [`tstl-plugin-reload-factorio-mod`](https://github.com/moeru-ai/airi-factorio/tree/main/packages/tstl-plugin-reload-factorio-mod): Reload Factorio mod when developing
- [🥺 SAD](https://github.com/moeru-ai/sad): Documentation and notes for self-host and browser running LLMs
- [`@velin-dev/ml`](https://github.com/luoling8192/velin): Use Vue SFC and Markdown to write easy to manage stateful prompts for LLM
- [`demodel`](https://github.com/moeru-ai/demodel): Easily boost the speed of pulling your models and datasets from various of inference runtimes.
- [`inventory`](https://github.com/moeru-ai/inventory): Centralized model catalog and default provider configurations backend service
- [MCP Launcher](https://github.com/moeru-ai/mcp-launcher): Easy to use MCP builder & launcher for all possible MCP servers, just like Ollama for models!
- ~~[`@proj-airi/elevenlabs`](https://github.com/moeru-ai/airi/tree/main/packages/elevenlabs): TypeScript definitions for ElevenLabs API~~
```mermaid
%%{ init: { 'flowchart': { 'curve': 'catmullRom' } } }%%
flowchart TD
Core("Core")
Unspeech["unspeech"]
DBDriver["@proj-airi/drizzle-duckdb-wasm"]
MemoryDriver["[WIP] Memory Alaya"]
DB1["@proj-airi/duckdb-wasm"]
ICONS["@proj-airi/lobe-icons"]
UI("@proj-airi/stage-ui")
Stage("Stage")
F_AGENT("Factorio Agent")
F_API["Factorio RCON API"]
F_MOD1["autorio"]
SVRT["@proj-airi/server-runtime"]
MC_AGENT("Minecraft Agent")
XSAI["xsai"]
subgraph Airi
DB1 --> DBDriver --> MemoryDriver --> Memory --> Core
ICONS --> UI --> Stage --> Core
Core --> STT
Core --> SVRT
end
STT --> |Speaking|Unspeech
SVRT --> |Playing Factorio|F_AGENT
SVRT --> |Playing Minecraft|MC_AGENT
subgraph Factorio Agent
F_AGENT --> F_API -..- factorio-server
subgraph factorio-server-wrapper
subgraph factorio-server
F_MOD1
end
end
end
subgraph Minecraft Agent
MC_AGENT --> Mineflayer -..- minecraft-server
subgraph factorio-server-wrapper
subgraph factorio-server
F_MOD1
end
end
end
XSAI --> Core
XSAI --> F_AGENT
XSAI --> MC_AGENT
```
```mermaid
%%{ init: { 'flowchart': { 'curve': 'catmullRom' } } }%%
flowchart TD
subgraph deploy&bundle
direction LR
HFUP["hfup"]
HF[/"HuggingFace Spaces"\]
HFUP -...- UI -...-> HF
HFUP -...- whisper-webgpu -...-> HF
HFUP -...- moonshine-web -...-> HF
end
```
## Models used
- [onnx-community/whisper-large-v3-turbo · Hugging Face](https://huggingface.co/onnx-community/whisper-large-v3-turbo)
## Similar Projects
### Open sourced ones
- [kimjammer/Neuro: A recreation of Neuro-Sama originally created in 7 days.](https://github.com/kimjammer/Neuro): very well completed implementation.
- [SugarcaneDefender/z-waif](https://github.com/SugarcaneDefender/z-waif): Great at gaming, autonomous, and prompt engineering
- [semperai/amica](https://github.com/semperai/amica/): Great at VRM, WebXR
- [elizaOS/eliza](https://github.com/elizaOS/eliza): Great examples and software engineering on how to integrate agent into various of systems and APIs
- [ardha27/AI-Waifu-Vtuber](https://github.com/ardha27/AI-Waifu-Vtuber): Great about Twitch API integrations
- [InsanityLabs/AIVTuber](https://github.com/InsanityLabs/AIVTuber): Nice UI and UX
- [IRedDragonICY/vixevia](https://github.com/IRedDragonICY/vixevia)
- [t41372/Open-LLM-VTuber](https://github.com/t41372/Open-LLM-VTuber)
- [PeterH0323/Streamer-Sales](https://github.com/PeterH0323/Streamer-Sales)
### Non-open-sourced ones
- https://clips.twitch.tv/WanderingCaringDeerDxCat-Qt55xtiGDSoNmDDr https://www.youtube.com/watch?v=8Giv5mupJNE
- https://clips.twitch.tv/TriangularAthleticBunnySoonerLater-SXpBk1dFso21VcWD
## Project Status
## Acknowledgements
- [pixiv/ChatVRM](https://github.com/pixiv/ChatVRM)
- [josephrocca/ChatVRM-js: A JS conversion/adaptation of parts of the ChatVRM (TypeScript) code for standalone use in OpenCharacters and elsewhere](https://github.com/josephrocca/ChatVRM-js)
- Design of UI and style was inspired by [Cookard](https://store.steampowered.com/app/2919650/Cookard/), [UNBEATABLE](https://store.steampowered.com/app/2240620/UNBEATABLE/), and [Sensei! I like you so much!](https://store.steampowered.com/app/2957700/_/), and artworks of [Ayame by Mercedes Bazan](https://dribbble.com/shots/22157656-Ayame) with [Wish by Mercedes Bazan](https://dribbble.com/shots/24501019-Wish)
- [mallorbc/whisper_mic](https://github.com/mallorbc/whisper_mic)
- [`xsai`](https://github.com/moeru-ai/xsai): Implemented a decent amount of packages to interact with LLMs and models, like [Vercel
## Star History
[](https://www.star-history.com/#moeru-ai/airi&Date)