https://github.com/affromero/splattie-widget
Interactive 3D Gaussian Splatting web component — like Rive/Lottie for 3D
https://github.com/affromero/splattie-widget
3dgs avatar flame gaussian-splatting interactive spark threejs typescript web-component
Last synced: 26 days ago
JSON representation
Interactive 3D Gaussian Splatting web component — like Rive/Lottie for 3D
- Host: GitHub
- URL: https://github.com/affromero/splattie-widget
- Owner: affromero
- License: mit
- Created: 2026-05-27T20:04:53.000Z (about 1 month ago)
- Default Branch: main
- Last Pushed: 2026-05-28T14:25:58.000Z (29 days ago)
- Last Synced: 2026-05-28T16:13:50.706Z (29 days ago)
- Topics: 3dgs, avatar, flame, gaussian-splatting, interactive, spark, threejs, typescript, web-component
- Language: TypeScript
- Homepage: https://splattie.app
- Size: 24.7 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# splattie-widget
**Interactive rigged 3D Gaussian Splatting - like Rive/Lottie for 3D**
[](https://www.npmjs.com/package/@afromero/splattie-widget)
[](LICENSE)
[](https://typescriptlang.org)
[](https://github.com/sparkjsdev/spark)
[](https://threejs.org)
[]()
[]()
[Quick Start](#quick-start) · [Format Spec](FORMAT.md) · [API](#api) · [Editor](#visual-editor) · [How It Works](#how-it-works)
---
A web component that makes Gaussian splats **reactive**. One file, one tag. **Heads** track the cursor with their eyes, blink, and emote on hover/click (FLAME rig); **bodies** turn their head and torso toward visitors and pose with two-bone arm IK (SMPL-X rig); **objects** use arbitrary skeletons and sparse LBS weights for cursor-follow and direct pose editing. 60fps, client-side. The widget branches on the bundle's `assetType` — same tag for heads, bodies, and objects.
**[See it live at afromero.co](https://afromero.co)** | **[Create your own at splattie.app](https://splattie.app)**
## Quick Start
```html
```
Or via npm:
```bash
npm install @afromero/splattie-widget
```
```typescript
import '@afromero/splattie-widget';
```
## The `.splattie` Format
> **v0.x experimental.** Core files (PLY, FLAME bones) follow established standards. Expression basis and states may evolve.
A ZIP bundle with a required `manifest.json` that declares every asset
and locks the file's `formatVersion` to the widget version. See
[`FORMAT.md`](FORMAT.md) for the full spec.
```
asset.splattie
├── manifest.json # (required) declares every asset + assetType + formatVersion
├── *.ply or *.spz # (required) Gaussian splats
│
│ # head (assetType: head) — FLAME rig:
├── bone_tree.json # (optional) Skeleton for skinning (5 FLAME bones)
├── lbs_weight_20k.json # (optional) Per-splat bone weights
├── expression_basis.bin # (optional) Blendshape basis
│
│ # body (assetType: body) — SMPL-X rig:
├── skeleton.json # (optional) 55-joint SMPL-X skeleton (baked-pose rest)
├── lbs_weights.json # (optional) Per-gaussian sparse LBS weights
│
│ # object (assetType: object) — arbitrary skeleton rig:
├── skeleton.json # (optional) Object joint hierarchy + rest positions
├── lbs_weights.bin # (optional) Binary sparse per-gaussian LBS weights
│
└── states.json # (optional) Interaction states
```
Splat data (.ply or .spz) - standard 3DGS format
Each splat has position, scale, rotation, opacity, and SH color. Auto-detected from file header. Works with any 3DGS method (LAM, DreamGaussian, InstantSplat, etc.). Standard format, unlikely to change.
bone_tree.json - skeleton hierarchy
5 FLAME bones: root > neck > jaw, leftEye, rightEye. Used for SplatSkinning (dual quaternion).
```json
{
"bones": [{
"name": "root",
"position": [x, y, z],
"children": [{
"name": "neck",
"position": [x, y, z],
"children": [
{ "name": "jaw", "position": [x, y, z] },
{ "name": "leftEye", "position": [x, y, z] },
{ "name": "rightEye", "position": [x, y, z] }
]
}]
}]
}
```
Stable structure. Bone names are conventions, not hard requirements. Without it: no eye tracking, no jaw animation.
lbs_weight_20k.json - per-splat bone weights
2D array `[num_splats][num_bones]`, each row sums to ~1.0. Widget selects top 4 per splat.
```json
[[0.8, 0.1, 0.05, 0.03, 0.02], ...]
```
Standard LBS format from FLAME. Without it: bones exist but nothing moves.
expression_basis.bin - FLAME blendshape basis
Per-splat position displacements for each expression coefficient. Moves all splats coherently for smile, lip shapes, etc.
```
Header: "EXPR" (4B) + num_vertices (u32 LE) + num_expressions (u32 LE)
Data: float32 LE array, shape (num_vertices, num_expressions, 3)
```
Optional sidecar `expression_basis.json` with semantic labels:
```json
{ "labels": ["jawDown", "lipsUp", "lipsL", ...], "num_expressions": 50 }
```
Experimental format, may add compression. Without it: bone-driven expressions still work.
skeleton.json + lbs_weights.json - SMPL-X body rig (bodies)
Bodies (`assetType: body`) use a 55-joint SMPL-X skeleton instead of FLAME bones:
```json
// skeleton.json — joints in the BAKED (photographed) rest pose
{ "rig": "smplx", "jointCount": 55, "names": ["Pelvis", "L_Hip", ...],
"parents": [-1, 0, 0, ...], "restPositions": [[x, y, z], ...] }
// lbs_weights.json — sparse top-K per-gaussian skinning
{ "numGaussians": 40000, "k": 4, "indices": [...], "weights": [...] }
```
The body is exported already posed (arms at rest), so the widget's rest pose is the
identity. From there it drives **head + torso look-at** toward the cursor and a
**two-bone arm IK** for editor posing, composing local joint rotations via SMPL-X
forward kinematics + linear blend skinning. Without these: the gaussians render but
don't articulate.
skeleton.json + lbs_weights.bin - object rig
Objects (`assetType: object`) use the same manifest-level LBS contract as bodies,
but with an arbitrary joint hierarchy:
```json
// skeleton.json
{ "rig": "puppeteer-object", "jointCount": 12, "names": ["root", "..."],
"parents": [-1, 0, ...], "restPositions": [[x, y, z], ...] }
// manifest excerpt
{ "weights": { "file": "lbs_weights.bin", "format": "lbsw-v1" } }
```
The binary weights file stores sparse uint16 joint indices and float16 weights for each
Gaussian. The widget projects terminal joints as editor handles, solves simple
joint-chain rotations when you drag a handle, and uses root/joint cursor-follow
settings for lightweight interactivity.
states.json - interaction state definitions
Each state (idle, hover, click) sets all 5 dimensions simultaneously.
```json
{
"defaults": {
"camera": { "theta": 0, "phi": 75, "radius": 0.5, "fov": 60 },
"autoBlink": { "interval": [2000, 7000], "duration": 150 }
},
"states": {
"idle": {
"ghost": { "amplitude": 0.003, "frequency": 0.4, "wobble": 0.2 },
"expression": { "jawOpen": 0, "smile": 0 },
"camera": { "theta": 0, "phi": 75, "radius": 0.5, "fov": 60 },
"rotation": [0, 0, 0],
"tracking": { "eyes": 1.0, "head": 0.1 }
},
"hover": { "..." : "..." },
"click": { "..." : "..." }
},
"transitions": {
"idle->hover": { "duration": 0.3, "easing": "ease-out" },
"*->click": { "duration": 0.1, "easing": "snap" }
}
}
```
Most likely to evolve. Without it: sensible defaults (eyes track, gentle float, auto-blink).
### Creating Your Own
**Visual editor**: `npm run dev`, adjust sliders, click "Download .splattie".
**From scratch**: ZIP a `.ply` with any combination of the optional files.
**From a photo or object image**: run the Splattie backend pipeline for heads
(LAM), bodies (LHM), or objects (TRELLIS + Puppeteer), then bundle the result.
Try it at [splattie.app](https://splattie.app).
## Five Dimensions of State
| Dimension | Controls | Example |
|-----------|----------|---------|
| **Ghost** | Floating/bobbing | Gentle hover on idle, freeze on click |
| **Expression** | FLAME blendshapes + bones | Smile on hover, surprise on click |
| **Camera** | Spherical position | Zoom in on hover |
| **Rotation** | Pitch/yaw/roll | Tilt head on hover |
| **Tracking** | Cursor-follow intensity | Heads: eyes/head. Bodies: head/torso. Objects: root/joints |
Interpolated between states with configurable easing and duration.
Expression system details
Two layers:
**Bone-driven** (SplatSkinning, 5 FLAME bones):
- Jaw open/close, neck pitch/yaw/roll
- Eye gaze direction (left/right, up/down)
- Brow raise/frown (left/right independently)
**Blendshape-driven** (FLAME expression basis, 10+ PCA coefficients):
- Moves all 20K splats coherently
- Smile, lip shapes, jaw articulation, cheek/nose deformation
- Spatial mask prevents beard/neck from deforming
## API
| Attribute | Description |
|-----------|-------------|
| `src` | URL to `.splattie` file (or `.ply`/`.spz`) |
| `background` | Background color hex (default: `#0e0e14`) |
| `width` | CSS width (default: `100%`) |
| `height` | CSS height (default: `400px`) |
```javascript
widget.addEventListener('splatload', () => {}); // ready
widget.addEventListener('splathover', () => {}); // cursor over asset
widget.addEventListener('splatclick', () => {}); // clicked asset
widget.addEventListener('splatleave', () => {}); // cursor left
widget.setState('hover'); // force transition
```
## Visual Editor
```bash
npm run dev # http://localhost:4002
```
Sliders for all 5 dimensions, camera sphere widget, state tabs with copy-forward, FLAME blendshape controls (heads), on-canvas IK drag handles to pose limbs (bodies), skeleton handles for object pose editing, drag-and-drop `.splattie` upload, export when done.
## How It Works
Built on [Spark 2.0](https://github.com/sparkjsdev/spark) (MIT, World Labs).
Architecture
1. **State machine** with per-dimension interpolation (lerp, slerp, ease curves)
2. **SplatSkinning** (dual quaternion) driving 5 FLAME bones from expression + cursor data (heads)
3. **SMPL-X skinning** (55-joint LBS) for bodies — head/torso look-at + analytic two-bone arm IK, composed via forward kinematics
4. **Generic object skinning** for arbitrary skeletons — root/joint cursor-follow, projected skeleton handles, and drag-to-pose chain solving
5. **Expression basis** - per-splat position offsets written to Spark's packed buffer (half-float, ~20K splats/frame)
6. **Hit detection** via `readPixels` after render (pixel-perfect)
7. **Auto-blink** with randomized interval and sine-curve via SplatEdit
8. **Gyroscope** tracking on mobile (iOS permission prompt included)
## Mobile
Touch + gyroscope. Eyes follow device orientation on mobile, touch position on tap. Return to center when finger lifts. iOS motion permission requested automatically.
## Browser Support
Chrome, Firefox, Safari, Edge. WebGL 2 required. No COOP/COEP headers needed.
## Acknowledgements
- [LAM](https://github.com/aigc3d/LAM) (SIGGRAPH 2025) - single-image 3DGS heads. Zixuan Zeng et al., AIGC3D team
- [LHM](https://github.com/aigc3d/LHM) (SIGGRAPH 2025) - single-image 3DGS bodies. AIGC3D team
- [TRELLIS](https://github.com/microsoft/TRELLIS) - single-image 3D asset reconstruction. Microsoft.
- [Puppeteer](https://github.com/snap-research/Puppeteer) - automatic skeleton and skinning for generated 3D assets. Snap Research.
- [FLAME](https://flame.is.tue.mpg.de/) - face model (heads). Tianye Li, Timo Bolkart, Michael J. Black, Hao Li, Javier Romero
- [SMPL-X](https://smpl-x.is.tue.mpg.de/) - body model (bodies). Pavlakos, Choutas, Ghorbani, Bolkart, Osman, Tzionas, Black (MPI)
- [Spark 2.0](https://github.com/sparkjsdev/spark) - World Labs (MIT)
- [3D Gaussian Splatting](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) - Kerbl, Kopanas, Leimkuhler, Drettakis (INRIA)
## License
MIT