https://github.com/nafisrayan/threejs-hand-control-panel

ThreeJS Mediapipe Handtracking
https://github.com/nafisrayan/threejs-hand-control-panel

hand-tracking mediapipe mr threejs vr webgl xr

Last synced: about 5 hours ago
JSON representation

ThreeJS Mediapipe Handtracking

• Host: GitHub
• URL: https://github.com/nafisrayan/threejs-hand-control-panel
• Owner: NafisRayan
• Created: 2024-03-03T18:31:47.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-03-05T20:58:43.000Z (9 months ago)
• Last Synced: 2024-03-06T19:54:59.192Z (9 months ago)
• Topics: hand-tracking, mediapipe, mr, threejs, vr, webgl, xr
• Language: JavaScript
• Homepage: https://nafisrayan.github.io/ThreeJS-Hand-Control-Panel/
• Size: 9.77 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: Readme.md

Awesome Lists containing this project

README

        
# ThreeJS Hand Control Panel

## Overview

The project is designed to create an interactive 3D visualization of hand landmarks detected in real-time from a video stream, using the MediaPipe Hands model and THREE.js for 3D rendering. It dynamically updates the positions of 3D objects representing hand landmarks based on the detected landmarks, adjusts the camera's position to follow the hand movements, and includes logic to recognize specific hand gestures. The visualization is rendered in a web environment, allowing users to see the hand landmarks in 3D space as they move their hands in front of the camera.

## Getting Started

To run the project, follow these steps:

1. Ensure you have a modern web browser installed.

2. Open the `index.html` file in your web browser.

3. Allow access to your camera when prompted.

## Project Structure

- `index.html`: The main HTML file that loads the project.

- `main.js`: The JavaScript file that contains the logic for the 3D visualization, including the setup of the THREE.js scene, the MediaPipe Hands model, and the logic for updating the 3D objects based on the detected hand landmarks.

## Dependencies

- **THREE.js**: A JavaScript library used for creating and displaying animated 3D computer graphics in a web browser.

- **MediaPipe Hands**: A machine learning model for detecting hand landmarks in real-time from a video stream.

## How It Works

1. The project uses the MediaPipe Hands model to detect hand landmarks in real-time from the video stream captured by the user's camera.

2. The detected hand landmarks are then used to update the positions of 3D objects in the THREE.js scene, creating a visual representation of the hand landmarks in 3D space.

3. The camera's position is dynamically adjusted to follow the hand movements, providing a more immersive experience.

4. Specific hand gestures are recognized, and the visualization responds accordingly, such as adding or removing objects from the scene.

## Contributing

Contributions are welcome! Please feel free to submit pull requests or open issues if you have any suggestions or improvements.