https://github.com/zunza5/handpointer
Hands-free computer control with MediaPipe. Real-time hand tracking maps an open hand to pointer movement and an index–middle finger touch to click; built with Python, MediaPipe, OpenCV, and PyAutoGUI for accessibility and HCI prototyping.
https://github.com/zunza5/handpointer
accessibility computer-vision gesture-recognition hand-tracking hands-free handtracking human-computer-interaction input-device mediapipe opencv pointer-control pyautogui python
Last synced: about 1 month ago
JSON representation
Hands-free computer control with MediaPipe. Real-time hand tracking maps an open hand to pointer movement and an index–middle finger touch to click; built with Python, MediaPipe, OpenCV, and PyAutoGUI for accessibility and HCI prototyping.
- Host: GitHub
- URL: https://github.com/zunza5/handpointer
- Owner: Zunza5
- Created: 2025-08-23T15:46:27.000Z (about 1 month ago)
- Default Branch: main
- Last Pushed: 2025-08-27T18:49:38.000Z (about 1 month ago)
- Last Synced: 2025-08-28T08:06:41.619Z (about 1 month ago)
- Topics: accessibility, computer-vision, gesture-recognition, hand-tracking, hands-free, handtracking, human-computer-interaction, input-device, mediapipe, opencv, pointer-control, pyautogui, python
- Language: Python
- Homepage:
- Size: 5.47 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# HandPointer
HandPuntator is a Python-based system that enables hands-free computer control using real-time hand tracking and gesture recognition powered by MediaPipe, translating finger landmarks into precise cursor movement and configurable actions like clicking, dragging, and scrolling for accessible, touch-free interaction scenarios .
Key contributions
- Implemented robust real-time hand landmark tracking and gesture mapping with MediaPipe Tasks to support low-latency, smooth pointer control and discrete actions .
- Designed a modular input layer that maps index-finger kinematics to cursor motion and pinch-orientation gestures to clicks and drags, making behaviors easy to extend or swap .
- Built an experimentation pipeline with OpenCV for video capture and visualization, enabling rapid iteration on gesture thresholds, smoothing, and UX tuning .
What it showcases
- Computer vision integration: applying MediaPipe’s Gesture Recognizer/Hands pipeline to interface design and human-computer interaction .
- Systems thinking: translating continuous hand pose data into stable, user-friendly controls through filtering, debouncing, and gesture state machines .
- Accessibility and HCI: enabling touch-free control for mobility and hygiene contexts, and prototyping alternative input methods .
## Gesture set
- Hand open: move the pointer by tracking the index fingertip position in real time, with smoothing to ensure stable motion.
- Index + middle finger touch: perform a left-click when the index fingertip contacts the middle fingertip, with a short debounce to prevent accidental multiple clicks.
## Run Locally
you will need python 3.12 to use
1: Clone the git repo and go to directory
```bash
git clone https://github.com/Zunza5/HandPointer.git
cd HandPointer
```
2: Create and activate a virtual environment
macOS/Linux:
```bash
python3.12 -m venv .venv
source .venv/bin/activate
```
windows:
```bash
python3.12 -m venv .venv
..venv\Scripts\Activate.ps1
```
3: Install dependencies
```bash
pip install -r requirements.txt
```
4: run main.py
```bash
python3.12 main.py
```
## Authors
- [@Zunza5](https://www.github.com/octokatherine)