Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ramezze/virtualmouse-handtracking
A python application to control mouse functions using hand gestures leveraging MediaPipe, OpenCV and Tensorflow.
https://github.com/ramezze/virtualmouse-handtracking
computer-vision handtracking handtrackingmodule kivy machine-learning mediapipe mediapipe-hands mouse multilayer-perceptron neural-network python scikit-learn scipy tensorflow virtualmouse webcam webcam-capture
Last synced: about 2 months ago
JSON representation
A python application to control mouse functions using hand gestures leveraging MediaPipe, OpenCV and Tensorflow.
- Host: GitHub
- URL: https://github.com/ramezze/virtualmouse-handtracking
- Owner: RamezzE
- License: mit
- Created: 2024-05-10T16:40:44.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2024-08-07T15:11:30.000Z (6 months ago)
- Last Synced: 2024-12-06T12:07:17.749Z (about 2 months ago)
- Topics: computer-vision, handtracking, handtrackingmodule, kivy, machine-learning, mediapipe, mediapipe-hands, mouse, multilayer-perceptron, neural-network, python, scikit-learn, scipy, tensorflow, virtualmouse, webcam, webcam-capture
- Language: Python
- Homepage:
- Size: 38 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Virtual Mouse
## Table of Contents
- [Overview](#overview)
- [App Preview](#app-preview)
- [Usage Preview](#usage-preview)
- [Methodology](#methodology)
- [Settings](#settings)
- [Prerequisites](#prerequisites)
- [Building](#building)
- [Acknowledgements](#acknowledgements)
- [License](#license)
## Overview
Built using OpenCV, MediaPipe, Tensorflow and Kivy for the user interface, Virtual Mouse allows mouse control using several hand gestures mapped to perform different mouse functions such as movement, dragging, clicking, scrolling, zooming etc. These mappings can be changed and configured along with additional settings in runtime through the Kivy application.
### App Preview
### Usage Preview
Mouse Movement
- Absolute Mouse:
- Relative Mouse:
Dragging
Clicking
Scrolling and Zooming
## Methodology
The Virtual Mouse app follows a structured methodology to achieve accurate hand gesture recognition and mapping to mouse actions for each frame.
The dataset used for training the model was manually recorded and is available in training/collected_dataset.zip.
## Settings
A simple guide to the available program's settings.
### Detection Settings
1. **Detection Confidence**: Sets the minimum confidence percentage for the detection to be considered successful.
- **High Detection Confidence**: Ensures that only highly confident hand detections are considered, which significantly reduces false detections but may miss more valid hand detections.
- **Low Detection Confidence**: Allows more detections to be considered valid, which can capture more hand movements but may include more false detections.
2. **Tracking Confidence**: Sets the minimum confidence percentage for the hand landmarks to be tracked successfully across frames.
- **High Tracking Confidence**: Ensures stable tracking of hand landmarks, which can result in smoother and more accurate gesture recognition but may lose track of hands more easily.
- **Low Tracking Confidence**: Allows for more continuous tracking of hand landmarks even with lower confidence, which can maintain tracking better but may introduce some jitter.
3. **Detection Responsiveness**: Adjusts how fast the program reacts to gesture changes. It has four values: Instant, Fast, Normal, and Slow.
- **Instant**: Program reacts immediately to gesture changes, providing the quickest response time but may be less stable.
- **Fast**: Program reacts quickly to gesture changes, balancing speed and stability.
- **Normal**: Program reacts at a moderate speed, providing a stable and responsive experience.
- **Slow**: Program reacts more slowly to gesture changes, prioritizing stability over speed.
### Mouse Settings
1. **Toggle Relative Mouse**: Toggle relative mouse mode on or off. This can be toggled using a gesture as well.
- **ON**: Introduces touch pad like behavior, moving the mouse relative to its previous position and set sensitivity. This works using multiple screens.
- **OFF**: Maps hand position in camera to screen position depending on screen size. This does not work using more than one screen.
2. **Relative Mouse Sensitivity**: Sets the mouse sensitivity to use when relative mouse mode is on.
3. **Scroll Sensitivity**: Sets the sensitivity or speed of scrolling when using the set gesture.
### Gesture Mappings
You can customize which gestures perform specific mouse actions (e.g., left click, right click, scroll, idle) through the settings in the Kivy application. This enables you to tailor the control scheme to your needs.
To modify gesture mappings:
1. Open the application and navigate to the settings screen.
2. Select the "Gesture Settings" option.
3. Change the mouse actions for the desired gestures.
## Prerequisites
- Python
- Pip
### Linux
- You can run these commands on linux to install python, pip, python-venv and other dependencies
```
sudo apt-get update
sudo apt install python3
sudo apt install python3-pip
sudo apt install python3-venv
sudo apt-get install python3-tk python3-dev
```
### Windows
- Install python from the official website
- Check if pip is already installed by running: `pip help`
- If pip is not installed, please check this [guide](https://www.geeksforgeeks.org/download-and-install-pip-latest-version/#windows) for installing pip on Windows
## Building
1. **Clone the repository**
```
git clone https://github.com/RamezzE/VirtualMouse-HandTracking.git
```
2. **Navigate to project folder**
```
cd VirtualMouse-HandTracking
```
3. **Create and activate a python virtual environment**
- Linux
```
python3 -m venv venv
source venv/bin/activate
```
- Windows
```
python -m venv venv
venv\Scripts\activate
```
4. **Install necessary pip packages**
```
pip install -r requirements.txt
```
- If the above command does not work or throws an error, run the below command instead
```
pip install numpy mediapipe scikit-learn kivy[base] mouse pyautogui pyaml opencv-python
```
- If you want to run training.ipynb, then install these extra packages as well
```
pip install ipykernel tensorflow pandas xgboost
```
5. **Run main application file**
```
python main.py
```
- Optionally, if you'd like to run the script directly without running the Kivy application, you can run the alternative main file
```
python main_no_gui.py
```
## Acknowledgements
Most of the icons used are provided by [Icons8](https://icons8.com/)
## License
This project is licensed under the [MIT License](LICENSE).