https://github.com/ishubhranshu/hand_gesture_control
Real-time hand gesture recognition using MediaPipe and TensorFlow for controlling mouse, scrolling, and task view with dynamic gestures.
https://github.com/ishubhranshu/hand_gesture_control
computer-vision deep-learning jupyter-notebook machine-learning neural-network project python
Last synced: about 1 month ago
JSON representation
Real-time hand gesture recognition using MediaPipe and TensorFlow for controlling mouse, scrolling, and task view with dynamic gestures.
- Host: GitHub
- URL: https://github.com/ishubhranshu/hand_gesture_control
- Owner: ishubhranshu
- Created: 2023-08-16T14:10:46.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2025-02-11T16:42:15.000Z (over 1 year ago)
- Last Synced: 2025-09-01T03:50:57.910Z (10 months ago)
- Topics: computer-vision, deep-learning, jupyter-notebook, machine-learning, neural-network, project, python
- Language: Jupyter Notebook
- Homepage:
- Size: 32.4 MB
- Stars: 0
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Hand Gesture Control: Real-Time Mouse and Scroll Controller
## Table of Contents
- [Introduction](#introduction)
- [Features](#features)
- [Demo](#demo)
- [File Description](#file-description)
- [Installation](#installation)
- [Usage](#usage)
- [Gesture Controls](#gesture-controls)
## Introduction
Welcome to **Hand Gesture Control**, a real-time hand gesture recognition system that allows you to control your computer's mouse and scrolling functionalities using simple hand gestures. Leveraging **MediaPipe** for landmark detection, **TensorFlow** for gesture classification, and **PyAutoGUI** for automating mouse and keyboard actions, this project provides an intuitive and seamless way to interact with your computer without the need for traditional input devices.
## Features
- **Real-Time Gesture Recognition**: Detects and classifies hand gestures in real-time using your webcam.
- **Mouse Movement Control**: Move the mouse cursor smoothly across the screen based on hand movements.
- **Click Actions**: Perform mouse clicks with specific gestures.
- **Smooth Scrolling**: Scroll up and down smoothly using designated gestures.
- **Task View Activation**: Trigger system-level actions like opening Task View with gestures.
- **Visual Feedback**: Displays predicted gestures and current actions on the video feed for better user awareness.
## Demo
*Figure: Demonstration of hand gestures controlling the mouse and scrolling actions.*
## Installation
### Prerequisites
- **Python 3.7 or higher**: Ensure you have Python installed. You can download it from [here](https://www.python.org/downloads/).
- **Webcam**: A functional webcam for capturing hand gestures.
### Clone the Repository
```bash
git clone https://github.com/ishubhranshu/hand_gesture_control.git
cd hand-gesture-control
```
## Files Description
### record_dataset.ipynb
- Captures hand gestures in real-time using a webcam and MediaPipe Hands, identifying and normalizing 21 key landmarks for gesture classification.
- Users assign gestures to specific classes (0–4) using keyboard inputs (e.g., pressing "1" for Class 0). The program collects up to 1000 samples per class.
- Saves all collected data to a CSV file, either creating a new file or appending to an existing one to enable incremental data collection.
### train_infer.ipynb
- Loads a CSV dataset containing hand gesture landmarks and their corresponding classes, splits the data into training and testing sets, and scales features using a StandardScaler.
- A neural network with two dense hidden layers and dropout is trained for gesture classification. The best model is saved using checkpoints and as a final .h5 file.
- The trained model is used to predict gestures in real-time via webcam feed, with gestures mapped to specific actions like mouse movement, clicks, and scrolling.