https://github.com/thedexplorer/svelte-vrm-live
Threlte Live – A SvelteKit + Three.js platform for live-streaming 3D VRM avatars. Features real-time chat with Google Generative AI, ElevenLabs TTS lip-sync, Mixamo animations, and Cloudflare edge deployment – perfect for creating interactive, high-performance streaming overlays.
https://github.com/thedexplorer/svelte-vrm-live
ai-agents ai-animation ai-anime ai-avatars ai-girlfriend elevenlabs lip-sync lipsync livestream mixamo-animations phoneme-conversion phonemes svelte sveltejs sveltekit threejs threlte tts vrm
Last synced: 3 months ago
JSON representation
Threlte Live – A SvelteKit + Three.js platform for live-streaming 3D VRM avatars. Features real-time chat with Google Generative AI, ElevenLabs TTS lip-sync, Mixamo animations, and Cloudflare edge deployment – perfect for creating interactive, high-performance streaming overlays.
- Host: GitHub
- URL: https://github.com/thedexplorer/svelte-vrm-live
- Owner: thedexplorer
- License: agpl-3.0
- Created: 2025-07-15T11:58:11.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2025-07-15T14:30:32.000Z (3 months ago)
- Last Synced: 2025-07-16T03:07:50.313Z (3 months ago)
- Topics: ai-agents, ai-animation, ai-anime, ai-avatars, ai-girlfriend, elevenlabs, lip-sync, lipsync, livestream, mixamo-animations, phoneme-conversion, phonemes, svelte, sveltejs, sveltekit, threejs, threlte, tts, vrm
- Language: TypeScript
- Homepage:
- Size: 30.3 MB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Roadmap: roadmap.md
Awesome Lists containing this project
README
# Threlte Live - 3D VRM Avatar Streaming Platform
A SvelteKit application for live-streaming 3D VRM avatars with AI-powered chat, text-to-speech, mixamo animations.
[Click to see showcase video](showcase.mp4)
## Features
- Threlte/Three.js for 3D rendering
- VRM avatar loading and animation with @pixiv/three-vrm
- Google Generative AI for conversational responses
- Text-to-speech with lip-sync
- Chat interface
- Mixamo animation integration
See [roadmap.md](roadmap.md) for full details and planned features.
### Text-to-Speech and Phonemes
The project uses ElevenLabs TTS with phoneme timings for VRM lip-sync.
Learn more:
- [What is a Phoneme](https://elevenlabs.io/blog/what-is-a-phoneme)
- [Prompting Controls](https://elevenlabs.io/docs/best-practices/prompting/controls)
Phonemes mapped: A, AA, AH, AE, AO, AW, AY, E, EH, ER, EY, I, IH, IY, O, OH, OW, OY, U, UH, UW, M, B, P, F, V, TH, L, R, NEUTRAL.
## Getting Started
### Prerequisites
- Node.js >= 18
- pnpm
### Installation
1. Clone the repository:
```bash
git clone https://github.com/yourusername/svelte-vrm-live.git
cd svelte-vrm-live
```
2. Install dependencies:
```bash
pnpm install
```
### Environment Variables
Set the following environment variables (for example, create a `.env` file in the project root):
```bash
# Google Generative AI
GOOGLE_API_KEY=your_google_api_key
# ElevenLabs Text-to-Speech
ELEVENLABS_API_KEY=your_elevenlabs_api_key
```
These keys are required for the chat and text-to-speech features.
## Developing
Start the development server:
```bash
pnpm dev
```
Open http://localhost:5173.
## Building
Build for production:
```bash
pnpm build
```
Preview:
```bash
pnpm run preview
```
## Keywords
svelte, sveltekit, threejs, threlte, vrm, 3d-avatar, ai-chat, text-to-speech, lipsync, phonemes, mixamo, animations, blockchain, solana, generative-ai, youtube-streaming
## Contributing
We welcome contributions! Before we can merge your first pull request you must sign our Contributor License Agreement (CLA). When you open a PR, GitHub will display a message from the CLA bot with a link to sign.
- Individual contributors: see [docs/CLA_INDIVIDUAL.md](docs/CLA_INDIVIDUAL.md)
- Companies/corporations: see [docs/CLA_CORPORATE.md](docs/CLA_CORPORATE.md)
Once the CLA is signed, the status check will update automatically, and we’ll be able to review your contribution.
## License
AGPL (see LICENSE file)