https://github.com/quiet-node/gesture-lab
A fun & cinematic 3D interaction laboratory. Features a high-fidelity Iron Man armor workshop, interactive galaxy simulations, and gamified gesture experiments. Powered by MediaPipe, Three.js, and TypeScript for on-device, latency-free tracking.
https://github.com/quiet-node/gesture-lab
Last synced: 9 days ago
JSON representation
A fun & cinematic 3D interaction laboratory. Features a high-fidelity Iron Man armor workshop, interactive galaxy simulations, and gamified gesture experiments. Powered by MediaPipe, Three.js, and TypeScript for on-device, latency-free tracking.
- Host: GitHub
- URL: https://github.com/quiet-node/gesture-lab
- Owner: quiet-node
- License: mit
- Created: 2025-12-07T08:45:25.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2026-03-10T21:44:02.000Z (3 months ago)
- Last Synced: 2026-03-10T22:58:24.191Z (3 months ago)
- Language: TypeScript
- Homepage: https://gesturelab.icu
- Size: 80.5 MB
- Stars: 157
- Watchers: 1
- Forks: 33
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# Gesture Lab
A laboratory for fun experiments with movement tracking and interactive visuals.
Live Demo: https://gesturelab.icu
> [!NOTE]
> **Disclaimer:** This is a fully vibe-coded project by Opus 4.5, Sonnet 4.5, and Gemini 3, and may not follow strict best practices. Use at your own discretion.
## 🛠️ Tech Stack
- [TypeScript](https://www.typescriptlang.org/) - Type-safe development
- [Three.js](https://threejs.org/) - 3D rendering engine
- [MediaPipe Tasks Vision](https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker) - Real-time hand tracking
- [inferencejs](https://github.com/roboflow/inferencejs) - Real-time client-side object detection
- [Vite](https://vitejs.dev/) - Next Generation Frontend Tooling
## 🧪 Experiments
### 1. Iron Man Workshop
Inspect and assemble the Iron Man Mk. III armor in a premium, holographic environment.
- **Gestures**: Open palm for exploded view, closed fist to assemble. Pinch to manipulate parts.
- [Design Document](docs/DESIGN-IRON-MAN-WORKSHOP.md)
https://github.com/user-attachments/assets/4b61baac-f71e-478a-bd0a-2509a7c9e2be
### 2. Cosmic Slash
Slice through cosmic objects with lightsaber hands in this high-energy arcade mode.
- **Gestures**: Slash with hands to destroy objects, build combos, and defeat bosses.
- [Design Document](docs/DESIGN-COSMIC-SLASH.md)
https://github.com/user-attachments/assets/6410fd2d-ea70-4f16-8bf9-df84b9cc1e59
### 3. Interactive Galaxy
Manipulate a universe of particles with your hands.
- **Gestures**: Move hands apart/together to resize, pinch for star bursts, close hands for Big Bang.
- [Design Document](docs/DESIGN-INTERACTIVE-GALAXY.md)
https://github.com/user-attachments/assets/70340864-a81a-4012-8bae-8f440271bbf7
### 4. Foggy Mirror
Clear the mist to reveal reality.
- **Gestures**: Wave hands to wipe the fog off the mirror.
- [Design Document](docs/DESIGN-FOGGY-MIRROR.md)
https://github.com/user-attachments/assets/65dfe3ac-6dc1-4339-be15-adaa11671228
### 5. Stellar Wave
Manipulate the fabric of space-time with an elegant, spring-based dot grid.
- **Gestures**:
- **Cosmic Pulse**: Right hand pinch to trigger ripples.
- **Force Field**: Left hand pinch for repulsion.
- **Gravity Well**: Both hands middle pinch for attraction.
- **Nebula Vortex**: Both hands ring pinch for rotation.
- **Quasar Surge**: Fist to charge, release for massive burst.
- **Cosmic Strings**: Both hands pinky pinch to pluck the grid.
- [Design Document](docs/DESIGN-STELLAR-WAVE.md)
### 6. Light Bulb
A physics-based simulation of an incandescent light bulb with realistic cord dynamics and thermal animation.
- **Gestures**:
- **Rotate**: Pinch to grab and rotate the bulb View.
- **Toggle**: Pull the cord to switch on/off.
- **Break**: Aggressive pulls snap the cord (Material Fatigue).
- [Design Document](docs/DESIGN-LIGHT-BULB.md)
https://github.com/user-attachments/assets/3373c54a-1d47-4bad-a508-3b8aca96e399
### 7. Magnetic Clutter
A satisfyingly chaotic digital toy where magnetic balls form clusters and react to energetic repulsion.
- **Gestures**:
- **Repulse**: Right hand fist to spawn a bright repulsor sphere and kick balls away.
- **Grab**: Left hand pinch to grab and manipulate individual balls.
- [Design Document](docs/DESIGN-MAGNETIC-CLUTTER.md)
https://github.com/user-attachments/assets/4e17cbdc-5474-4f81-8400-16b7f9376cb0
### 8. Voxel Builder
A spatial computing voxel editor where you sculpt 3D models with your hands in real-time.
- **Gestures**:
- **Sculpt**: Right hand pinch to draw voxels.
- **Erase**: Left hand fist modifies draw mode to delete voxels.
- **Turntable**: Left hand pinch to rotate the sculpture.
- **Palette**: Pinky pinch to cycle color themes (Spectrum, Cyberpunk, etc.).
- [Design Document](docs/DESIGN-VOXEL-BUILDER.md)
https://github.com/user-attachments/assets/abf2ef4f-428b-492a-8b06-7970b4a1f5d7
### 9. Visual Recognition
A real-time object detection experiment that runs entirely inside your browser for maximum privacy and performance.
- **Capabilities**: Recognizes 80 common objects (people, phones, laptops, etc.) using the COCO dataset.
- **Tech**: YOLOv8n (Nano) architecture with WebGL GPU acceleration and background Web Worker isolation.
- [Design Document](docs/DESIGN-VISUAL-RECOGNITION.md)
https://github.com/user-attachments/assets/f0b4495b-e250-4b60-851b-c0cb44451326
## 🚀 Quick Start
```bash
# Install dependencies
bun install
# Start development server
bun dev
# Start preview
bun run build && bun preview
# validate build
bun validate-build
```
## 🎮 Controls
- **I**: Switch to Iron Man Workshop Mode
- **C**: Switch to Cosmic Slash Mode
- **G**: Switch to Galaxy Mode
- **F**: Switch to Foggy Mirror Mode
- **S**: Switch to Stellar Wave Mode
- **L**: Switch to Light Bulb Mode
- **K**: Switch to Magnetic Clutter Mode
- **V**: Switch to Voxel Builder Mode
- **O**: Switch to Visual Recognition Mode
- **H**: Toggle Hints
- **D**: Toggle Debug Panel
- **Esc**: Stop/Dispose
---
_Vibed by [@quiet_node](https://x.com/quiet_node)_