{"id":28579161,"url":"https://github.com/khaledalam/avatoon","last_synced_at":"2026-07-09T14:00:29.397Z","repository":{"id":292141399,"uuid":"979339593","full_name":"khaledalam/avatoon","owner":"khaledalam","description":"React Three Fiber component for animated avatars with lip-sync and head motion.","archived":false,"fork":false,"pushed_at":"2026-07-09T08:06:57.000Z","size":32947,"stargazers_count":12,"open_issues_count":2,"forks_count":7,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-07-09T08:07:39.586Z","etag":null,"topics":["3d","animation","avatar","avatars","drei","glb","lip-sync","morph-targets","npm","npm-package","r3f","react","react-three-fiber","text-to-speech","three","three-js","threejs","typescript","visemes","webgl"],"latest_commit_sha":null,"homepage":"https://khaledalam.net/avatoon/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/khaledalam.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-05-07T11:11:59.000Z","updated_at":"2026-07-09T08:07:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"3b39a5b5-9a1b-4784-bf6f-9db5c125e482","html_url":"https://github.com/khaledalam/avatoon","commit_stats":null,"previous_names":["khaledalam/avatoon"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/khaledalam/avatoon","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khaledalam%2Favatoon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khaledalam%2Favatoon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khaledalam%2Favatoon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khaledalam%2Favatoon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/khaledalam","download_url":"https://codeload.github.com/khaledalam/avatoon/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khaledalam%2Favatoon/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35301501,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-09T02:00:07.329Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","animation","avatar","avatars","drei","glb","lip-sync","morph-targets","npm","npm-package","r3f","react","react-three-fiber","text-to-speech","three","three-js","threejs","typescript","visemes","webgl"],"created_at":"2025-06-11T01:38:28.140Z","updated_at":"2026-07-09T14:00:29.362Z","avatar_url":"https://github.com/khaledalam.png","language":"TypeScript","funding_links":["https://github.com/sponsors/khaledalam","https://www.buymeacoffee.com/khaledalam","https://paypal.me/khaledalam"],"categories":[],"sub_categories":[],"readme":"# 🧠 Avatoon\n\n\n[![npm version][npm-img]][npm-url]\n[![npm downloads][downloads-img]][npm-url]\n[![minzipped size][size-img]][size-url]\n[![Build Status][build-img]][build-url]\n[![Code Coverage][codecov-img]][codecov-url]\n[![TypeScript][types-img]][npm-url]\n[![License][license-img]][license-url]\n[![GitHub stars][stars-img]][stars-url]\n[![Sponsor][sponsor-img]][sponsor-url]\n\n\n**Avatoon** is a lightweight React Three Fiber component for rendering animated 3D avatars with real-time viseme-driven lip-sync. It supports lifelike head movements, morph target control, and optional goal-based gestures (e.g., flexing, sleeping), making it ideal for voice assistants, interactive characters, or storytelling apps.\n\n### 🎮 [**Try the live demo →**](https://khaledalam.net/avatoon/)\n\n[![Open in StackBlitz](https://developer.stackblitz.com/img/open_in_stackblitz.svg)](https://stackblitz.com/github/khaledalam/avatoon/tree/main/example)\n\n\u003cimg src=\"https://raw.githubusercontent.com/khaledalam/avatoon/main/test/assets/avatoon.gif\" /\u003e\n\n[demo video](https://youtu.be/9ODqu2EZvtw)\n\n\n## ✨ Features\n- 🎤 Real-time lip-sync using phoneme-viseme mapping  \n- 🧍 Subtle head motion animation while talking  \n- 👁️ Automatic eye-blinking (when the model exposes blink morph targets)  \n- 🎛️ Imperative `play()` / `stop()` control via a `ref`  \n- 🌐 GLTF model support via `useGLTF`  \n- ⚛️ Plug-and-play with React Three Fiber + Drei  \n- 🎯 Goal-based gestures like \"Muscle\" or \"Sleep\"\n\n## 📦 Installation\n\n```bash\nnpm install avatoon\n```\n\n### Requirements (peer dependencies)\n\nAvatoon builds on React and the React Three Fiber ecosystem. Install these\nalongside it if your project doesn't already have them:\n\n```bash\nnpm install react react-dom three @react-three/fiber @react-three/drei\n```\n\n| Peer dependency       | Supported version |\n| --------------------- | ----------------- |\n| `react` / `react-dom` | `\u003e=18`            |\n| `three`               | `\u003e=0.153.0`       |\n| `@react-three/fiber`  | `\u003e=8.0.0`         |\n| `@react-three/drei`   | `\u003e=9.0.0`         |\n\n\u003e Runs in any browser with WebGL support. Written in TypeScript — type\n\u003e definitions ship with the package.\n\n## 🚀 Usage\n\n### `Avatoon` — full avatar with audio-synced lip-sync\n\n```jsx\nimport { Avatoon } from \"avatoon\";\n\nconst visemeJson = {\n  visemes: [\n    { time: 0, viseme: \"X\" },\n    { time: 1.3, viseme: \"A\" },\n    { time: 1.367, viseme: \"C\" },\n  ],\n  audio_base64: \"\", // base64-encoded WAV (optional)\n};\n\nexport default function App() {\n  return (\n    \u003cdiv style={{ width: \"100%\", height: 400 }}\u003e\n      \u003cAvatoon\n        glbUrl=\"https://raw.githubusercontent.com/khaledalam/avatoon/main/test/assets/placeholder-avatar.glb\"\n        goal=\"Normal\"\n        visemeJson={visemeJson}\n        showPlayVoiceButton\n        onRenderComplete={() =\u003e console.log(\"Render Completed!\")}\n      /\u003e\n    \u003c/div\u003e\n  );\n}\n```\n\n### `LipSyncAvatoon` — standalone lip-sync (no audio required)\n\nA self-contained component with a built-in Start/Stop button that animates\nprocedural mouth movement. Great for quick demos or \"talking head\" idle states.\n\n```jsx\nimport { LipSyncAvatoon } from \"avatoon\";\n\nexport default function App() {\n  return (\n    \u003cdiv style={{ width: \"100%\", height: 400 }}\u003e\n      \u003cLipSyncAvatoon glbUrl=\"/avatar.glb\" /\u003e\n    \u003c/div\u003e\n  );\n}\n```\n\n### Imperative control (ref)\n\nDrive playback from your own UI instead of the built-in button by passing a\n`ref` — it exposes `play()`, `stop()`, and `toggle()`:\n\n```jsx\nimport { useRef } from \"react\";\nimport { Avatoon, type AvatoonHandle } from \"avatoon\";\n\nfunction App() {\n  const avatar = useRef\u003cAvatoonHandle\u003e(null);\n\n  return (\n    \u003c\u003e\n      \u003cbutton onClick={() =\u003e avatar.current?.play()}\u003eSpeak\u003c/button\u003e\n      \u003cbutton onClick={() =\u003e avatar.current?.stop()}\u003eStop\u003c/button\u003e\n      \u003cdiv style={{ width: \"100%\", height: 400 }}\u003e\n        \u003cAvatoon ref={avatar} glbUrl=\"/avatar.glb\" visemeJson={visemeJson} /\u003e\n      \u003c/div\u003e\n    \u003c/\u003e\n  );\n}\n```\n\n### To run the example app locally:\n```bash\nnpm run example\n```\n\n## 🧩 API\n\n### `Avatoon(props)`\n\n| Prop                  | Type         | Default      | Description                                            |\n| --------------------- | ------------ | ------------ | ------------------------------------------------------ |\n| `glbUrl`              | `string`     | *(required)* | URL to the `.glb` avatar file (T1 or T2)               |\n| `goal`                | `AvatoonGoal`| `\"Normal\"`   | Motion preset: `\"Normal\"`, `\"Muscle\"`, or `\"Sleep\"`    |\n| `onRenderComplete`    | `() =\u003e void` | `undefined`  | Callback fired when avatar finishes rendering          |\n| `onError`             | `(error: Error) =\u003e void` | `undefined` | Fired if the model fails to load (bad `glbUrl`) instead of crashing |\n| `visemeJson`          | `VisemeData` | `undefined`  | JSON structure for syncing visemes with audio playback |\n| `showPlayVoiceButton` | `boolean`    | `false`      | If true, renders a play/stop voice button in the scene |\n| `fov`                 | `number`     | `24`         | Camera vertical field-of-view                          |\n| `cameraPosition`      | `[number, number, number]` | `[0, 1.45, 2.3]` | Camera position `[x, y, z]`             |\n| `cameraTarget`        | `[number, number, number]` | `[0, 1.35, 0]`   | OrbitControls look-at target `[x, y, z]` |\n| `environmentPreset`   | `EnvironmentPreset` | `\"sunset\"` | drei lighting preset (`city`, `dawn`, `night`, …) |\n| `environmentFiles`    | `string \\| string[]` | `undefined` | Custom HDR/EXR file(s); overrides the preset |\n| `environmentBackground` | `boolean`  | `false`      | Render the environment as the scene background |\n\n\u003e Tune `fov` / `cameraPosition` / `cameraTarget` to frame your own avatar model.\n\u003e You can also drive playback imperatively via a [`ref`](#imperative-control-ref).\n\n### `LipSyncAvatoon(props)`\n\nA lightweight, self-contained variant that drives procedural mouth movement\n(no `visemeJson` or audio needed) and renders its own Start/Stop button.\n\n| Prop             | Type     | Default        | Description                          |\n| ---------------- | -------- | -------------- | ------------------------------------ |\n| `glbUrl`         | `string` | `\"/avatar.glb\"` | URL to the `.glb` avatar file (T2)  |\n| `fov`            | `number` | `24`           | Camera vertical field-of-view        |\n| `cameraPosition` | `[number, number, number]` | `[0, 1.45, 2.3]` | Camera position `[x, y, z]`  |\n| `cameraTarget`   | `[number, number, number]` | `[0, 1.35, 0]`   | OrbitControls look-at target |\n| `onError`        | `(error: Error) =\u003e void` | `undefined` | Fired if the model fails to load |\n| `environmentPreset` / `environmentFiles` / `environmentBackground` | — | — | Same environment options as `Avatoon` |\n\n### Exposed building blocks\n\nFor advanced composition you can also import the lower-level pieces:\n`AvatoonModel`, `CameraFovAnimator`, `SceneEnvironment`, and `AvatoonErrorBoundary`.\n\n## 👤 Avatar Types\n- T1 (Static Face - Realistic)\n    - Most photorealistic\n    - No facial morphing\n    - Lightweight\n    - \u003cimg src=\"https://raw.githubusercontent.com/khaledalam/avatoon/main/test/assets/V1_bg.jpg\" width=\"100\"/\u003e\n- T2 (Blendshape Face - Expressive)\n  - Separate eyeballs and mouth\n  - Supports morph targets / ARKit visemes\n  - Slightly less realistic but animatable\n  - \u003cimg src=\"https://raw.githubusercontent.com/khaledalam/avatoon/main/test/assets/V2_bg.jpg\" width=\"100\"/\u003e\n\n## 📘 Types\n\n```ts\ninterface VisemeData {\n  visemes: Array\u003c{ time: number; viseme: string | null }\u003e;\n  audio_base64?: string;\n}\n```\n\n## 🎚️ Generating viseme data\n\n`visemeJson` is what drives the mouth animation. Each entry pairs a **`time`**\n(in seconds, from the start of the audio) with a single-letter **viseme code**.\nCodes are mapped onto the model's morph targets; unknown or silent codes simply\nrest the mouth.\n\n| Code       | Mouth shape / example sound   |\n| ---------- | ----------------------------- |\n| `A`        | open — \"**a**\" as in *apple*  |\n| `B`        | lips together — *p, b, m*     |\n| `C`        | *ch, sh, j*                   |\n| `D`        | *d, t, th*                    |\n| `E`        | \"*eh*\", \"*ae*\"                |\n| `F`        | *f, v*                        |\n| `G`        | *g, k*                        |\n| `I`        | \"*ee*\"                        |\n| `J`        | *r, l, y*                     |\n| `K`        | \"*oo*\", \"*u*\"                 |\n| `H` / `X`  | silence / rest                |\n\nThe standard **Oculus / Ready Player Me viseme names** are also accepted as codes\ndirectly — `sil`, `PP`, `FF`, `TH`, `DD`, `kk`, `CH`, `SS`, `nn`, `RR`, `aa`,\n`E`, `I`, `O`, `U` — which is what the converter helpers below emit.\n\nEntries must be ordered by ascending `time`. Optionally provide the spoken audio\nas a base64-encoded WAV via `audio_base64` to play it in sync (used when\n`showPlayVoiceButton` is enabled).\n\n```jsonc\n{\n  \"visemes\": [\n    { \"time\": 0.00, \"viseme\": \"X\" },\n    { \"time\": 0.12, \"viseme\": \"B\" },\n    { \"time\": 0.20, \"viseme\": \"A\" },\n    { \"time\": 0.35, \"viseme\": \"I\" }\n  ],\n  \"audio_base64\": \"UklGR... (optional WAV)\"\n}\n```\n\n### From a speech engine\n\nRather than hand-authoring the timeline, generate it from a TTS/lip-sync engine.\nAvatoon ships converters for the most common sources:\n\n```js\nimport {\n  Avatoon,\n  fromAzureVisemes,   // Azure Speech SDK viseme events\n  fromPollySpeechMarks, // AWS Polly speech marks\n  fromRhubarb,        // Rhubarb Lip Sync JSON\n} from \"avatoon\";\n\n// Azure: collect events from the Speech SDK's `visemeReceived` callback\nconst visemeJson = fromAzureVisemes(azureEvents); // [{ visemeId, audioOffset }]\n\n// AWS Polly: pass the `viseme` speech marks\nconst visemeJson = fromPollySpeechMarks(pollyMarks); // [{ time, value, type }]\n\n// Rhubarb: pass its JSON output (or just the mouthCues array)\nconst visemeJson = fromRhubarb(rhubarbOutput); // { mouthCues: [{ start, end, value }] }\n```\n\nEach returns a `VisemeData` object ready to hand to `\u003cAvatoon visemeJson={...} /\u003e`.\nTimes are normalized to seconds. Add `audio_base64` yourself if you want synced\nplayback.\n\n\u003e **Model requirement (T2):** audio-synced lip-sync needs a `.glb` whose mesh\n\u003e exposes `viseme_*` morph targets (ARKit / Oculus / Ready Player Me naming —\n\u003e e.g. `viseme_aa`, `viseme_PP`, `viseme_CH`). T1 models have no morph targets,\n\u003e so use `LipSyncAvatoon` or head-motion only.\n\n\n## 🤝 Contribution\nPull requests are welcome! See [CONTRIBUTING.md](./CONTRIBUTING.md) for local\nsetup and development workflow, and please review our\n[Code of Conduct](./CODE_OF_CONDUCT.md). To report a security issue, see\n[SECURITY.md](./SECURITY.md).\n\n\n## 💖 Support\n\nIf Avatoon is useful to you, consider supporting its development — it helps a lot\nand keeps the project maintained:\n\n- ⭐ **Star the repo** — the easiest way to help\n- 💛 [**Sponsor on GitHub**](https://github.com/sponsors/khaledalam)\n- ☕ [**Buy Me a Coffee**](https://www.buymeacoffee.com/khaledalam)\n- 💵 [**PayPal**](https://paypal.me/khaledalam)\n\n\n## 📄 License\n\n[MIT](./LICENSE) © [Khaled Alam](https://khaledalam.net/)\n\n\n## 🛡️ Author\n\n**Khaled Alam**\n\n📧 [khaledalam.net@gmail.com](mailto:khaledalam.net@gmail.com)\u003cbr /\u003e\n🌍 [Website](https://khaledalam.net/) | [LinkedIn](https://www.linkedin.com/in/khaledalam/) | [X(Twitter)](https://x.com/khaledalamxyz)\n\n[build-img]:https://github.com/khaledalam/avatoon/actions/workflows/release.yml/badge.svg\n[build-url]:https://github.com/khaledalam/avatoon/actions/workflows/release.yml\n[npm-img]:https://img.shields.io/npm/v/avatoon\n[npm-url]:https://www.npmjs.com/package/avatoon\n[downloads-img]:https://img.shields.io/npm/dm/avatoon\n[size-img]:https://img.shields.io/bundlejs/size/avatoon\n[size-url]:https://bundlejs.com/?q=avatoon\n[types-img]:https://img.shields.io/npm/types/avatoon\n[license-img]:https://img.shields.io/npm/l/avatoon\n[license-url]:https://github.com/khaledalam/avatoon/blob/main/LICENSE\n[stars-img]:https://img.shields.io/github/stars/khaledalam/avatoon?style=flat\n[stars-url]:https://github.com/khaledalam/avatoon/stargazers\n[sponsor-img]:https://img.shields.io/badge/%E2%9D%A4-Sponsor-ea4aaa\n[sponsor-url]:https://github.com/sponsors/khaledalam\n[codecov-img]:https://codecov.io/gh/khaledalam/avatoon/branch/main/graph/badge.svg\n[codecov-url]:https://codecov.io/gh/khaledalam/avatoon","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkhaledalam%2Favatoon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkhaledalam%2Favatoon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkhaledalam%2Favatoon/lists"}