https://github.com/collidingscopes/3d-model-playground
Control 3D models using hand gestures and voice commands in real-time. Threejs / mediapipe computer vision
https://github.com/collidingscopes/3d-model-playground
3d-model animation augmented-reality computer-vision hand-tracking mediapipe open-source rosebud threejs tutorial voice-recognition web-speech-api
Last synced: 25 days ago
JSON representation
Control 3D models using hand gestures and voice commands in real-time. Threejs / mediapipe computer vision
- Host: GitHub
- URL: https://github.com/collidingscopes/3d-model-playground
- Owner: collidingScopes
- Created: 2025-06-03T11:48:45.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2025-06-22T18:08:18.000Z (3 months ago)
- Last Synced: 2025-06-22T18:27:49.703Z (3 months ago)
- Topics: 3d-model, animation, augmented-reality, computer-vision, hand-tracking, mediapipe, open-source, rosebud, threejs, tutorial, voice-recognition, web-speech-api
- Language: JavaScript
- Homepage: https://collidingscopes.github.io/3d-model-playground/
- Size: 2.16 MB
- Stars: 163
- Watchers: 2
- Forks: 17
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
Awesome Lists containing this project
README
# 3D Model Playground
Control 3D models using hand gestures and voice commands in real-time.
An interactive web app built with threejs, mediapipe computer vision, web speech API, and rosebud AI.
- Say "drag", "rotate", "scale", or "animate" to change the interaction mode
- Pinch fingers to control the 3D model
- Drag/drop a new 3D model onto the page to import it (GLB/GLTF format)
[Video](https://youtu.be/_I1E44Fp1Es?si=lR2otqR_-ZGdIGXT) | [Live Demo](https://collidingscopes.github.io/3d-model-playground/)
## Requirements
- Modern web browser with WebGL support
- Camera / microphone access
## Technologies
- **Three.js** for 3D rendering
- **MediaPipe** for hand tracking and gesture recognition
- **Web Speech API** for speech recognition
- **HTML5 Canvas** for visual feedback
- **JavaScript** for real-time interaction
## Setup for Development
```bash
# Clone this repository
git clone https://github.com/collidingScopes/3d-model-playground
# Navigate to the project directory
cd 3d-model-playground
# Serve with your preferred method (example using Python)
python -m http.server
```
Then navigate to `http://localhost:8000` in your browser.
## License
MIT License
## Credits
- Three.js - https://threejs.org/
- MediaPipe - https://mediapipe.dev/
- Rosebud AI - https://rosebud.ai/
- Quaternius 3D models - https://quaternius.com/
## Related Projects
I've released several computer vision projects (with code + tutorials) here:
[Fun With Computer Vision](https://www.funwithcomputervision.com/)
You can purchase lifetime access and receive the full project files and tutorials. I'm adding more content regularly :)
You might also like some of my other open source projects:
- [Particular Drift](https://collidingScopes.github.io/particular-drift) - Turn photos into flowing particle animations
- [Liquid Logo](https://collidingScopes.github.io/liquid-logo) - Transform logos and icons into liquid metal animations
- [Video-to-ASCII](https://collidingScopes.github.io/ascii) - Convert videos into ASCII pixel art
## Contact
- Instagram: [@stereo.drift](https://www.instagram.com/stereo.drift/)
- Twitter/X: [@measure_plan](https://x.com/measure_plan)
- Email: [stereodriftvisuals@gmail.com](mailto:stereodriftvisuals@gmail.com)
- GitHub: [collidingScopes](https://github.com/collidingScopes)
## Donations
If you enjoyed this, feel free to buy me a coffee.
My name is Alan, and I enjoy building open source software for computer vision, games, and more. This would be much appreciated during late-night coding sessions!
[](https://www.buymeacoffee.com/stereoDrift)