{"id":26150516,"url":"https://github.com/uni-creator/handgestureautomation","last_synced_at":"2026-04-07T16:31:06.957Z","repository":{"id":272591200,"uuid":"917117119","full_name":"Uni-Creator/HandGestureAutomation","owner":"Uni-Creator","description":"A Python-based application that lets you control your computer mouse using hand gestures captured through your webcam. Using computer vision and hand tracking technology, it enables natural mouse movements, clicks, scrolling, and drag operations without touching your physical mouse.","archived":false,"fork":false,"pushed_at":"2026-02-18T12:11:53.000Z","size":31,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-18T16:39:01.941Z","etag":null,"topics":["ai","automation","computer-vision","hand-gesture-recognition","mediapipe","mouse-control","opencv","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Uni-Creator.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-15T11:41:55.000Z","updated_at":"2026-02-18T12:11:56.000Z","dependencies_parsed_at":"2025-01-15T13:36:02.379Z","dependency_job_id":"70794c79-b248-4ebc-982a-2b248073581a","html_url":"https://github.com/Uni-Creator/HandGestureAutomation","commit_stats":null,"previous_names":["uni-creator/handgestureautomation"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Uni-Creator/HandGestureAutomation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Uni-Creator%2FHandGestureAutomation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Uni-Creator%2FHandGestureAutomation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Uni-Creator%2FHandGestureAutomation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Uni-Creator%2FHandGestureAutomation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Uni-Creator","download_url":"https://codeload.github.com/Uni-Creator/HandGestureAutomation/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Uni-Creator%2FHandGestureAutomation/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31520388,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-07T16:28:08.000Z","status":"ssl_error","status_checked_at":"2026-04-07T16:28:06.951Z","response_time":105,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","automation","computer-vision","hand-gesture-recognition","mediapipe","mouse-control","opencv","python"],"created_at":"2025-03-11T05:59:55.263Z","updated_at":"2026-04-07T16:31:06.940Z","avatar_url":"https://github.com/Uni-Creator.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🖱️ Hand Gesture Mouse Controller\n\nControl your computer mouse entirely with hand gestures — no physical mouse needed.  \nUses **MediaPipe** for real-time hand tracking and **OpenCV** for camera input, with three interchangeable mouse-control backends.\n\n---\n\n##  Features\n\n| Feature | Description |\n|---|---|\n| **Move Cursor** | Precise, smoothed cursor movement using your index finger |\n| **Left Click** | Half-fold your index finger to click |\n| **Right Click** | Raise index + middle fingers, then bring index tip close to pinky base |\n| **Drag \u0026 Drop** | Hold thumb + pinky up to drag; release to drop |\n| **Scroll** | Raise index + middle + ring fingers and move hand up/down |\n| **Orientation Guard** | Ignores input when hand is turned or flipped (palm facing away) |\n| **Smooth Movement** | Configurable smoothening factor prevents jitter |\n\n---\n## Demo\n\nWatch a demo of the hand gesture mouse controller in action:\n\n\n\nhttps://github.com/user-attachments/assets/e914f0d4-db16-4bc7-b61d-b92f7e13c1c5\n\n\n\n## Gesture Reference\n\n\u003e **Finger array format:** `[Thumb, Index, Middle, Ring, Pinky]`  \n\u003e `1` = finger extended/up · `0` = finger folded/down  \n\u003e The array represents what `fingers_up()` returns for each gesture.\n\n### Move Cursor\n```\n[0, 1, 0, 0, 0]  — only index finger up (fully extended, not half-folded)\n```\nPoint your **index finger** at the camera. The cursor follows your fingertip within the active zone (the pink rectangle on screen).\n\n---\n\n### Left Click\n```\nhalf-closed: [0, 1, 0, 0, 0]  — index finger half-folded (bent at middle joint)\n```\nSlightly **curl your index finger** (don't fully fold it). The system detects the half-closed state separately from a fully extended finger, so moving and clicking don't conflict.\n\n---\n\n### Right Click\n```\nfingers_up → [0, 1, 1, 0, 0]  — index + middle fingers up\n+ index fingertip brought close to pinky base (landmark 17)\n```\nRaise your **index and middle fingers** together, then **pinch the index tip toward the base of your pinky**. The click fires when the distance between those two points drops below 30 px.\n\n---\n\n### Drag \u0026 Drop\n```\n[0, 1, 0, 0, 1]  — index + pinky up, rest folded\n```\nRaise your **index finger and pinky** while keeping the other fingers down. The left mouse button is held as long as this gesture is held. Release (change gesture) to drop.\n\n---\n\n### Scroll\n```\n[0, 1, 1, 1, 0]  — index + middle + ring fingers up\n```\nRaise your **index, middle, and ring fingers** together. Move your hand **up** to scroll up and **down** to scroll down. The scroll speed is proportional to movement speed.\n\n---\n\n### No Action (hand fully open — safety pose)\n```\n[1, 1, 1, 1, 1]  — all fingers up\n```\nAll five fingers fully extended. The system **ignores** this pose so you can rest your hand in a natural open position without triggering any action.\n\n---\n\n## Backend Comparison\n\nThree mouse-control backends are provided. They differ in how they send input events to the OS:\n\n| Backend | File | Accuracy | Latency | Platform | Notes |\n|---|---|---|---|---|---|\n| **pyautogui** | `mouse_control_pyautogui.py` | ⭐⭐ | Higher | Cross-platform | Easiest to install; uses screenshot-based failsafe; slowest of the three |\n| **pynput** | `mouse_control_pynput.py` | ⭐⭐⭐ | Medium | Cross-platform | Good balance; works well on most systems |\n| **Win32 API** | `mouse_control_win32.py` | ⭐⭐⭐⭐ | Lowest | **Windows only** | Sends raw Win32 mouse events; most responsive and accurate; **recommended on Windows** |\n\n**Ranking: `pyautogui` \u003c `pynput` ≈ `Win32`**\n\n\u003e `Win32` is the most accurate because it bypasses Python-level abstractions and calls the OS mouse event API directly. `pynput` is close behind and works cross-platform. `pyautogui` adds extra overhead (screenshot-based coordinate mapping, built-in delays) making it the least responsive.\n\n---\n\n## Project Structure\n\n```\nHandGestureAutomation/\n│\n├── hand_tracker.py              # Core hand-tracking module (MediaPipe wrapper)\n│                                #   → HandDetector class: find_hands, find_position,\n│                                #     fingers_up, fingers_half_closed, find_distance,\n│                                #     is_hand_flipped, is_hand_turned\n│\n├── mouse_control_win32.py       # Recommended — Win32 API backend (Windows only)\n├── mouse_control_pynput.py      # pynput backend (cross-platform)\n├── mouse_control_pyautogui.py   # pyautogui backend (cross-platform, slower)\n│\n├── hand_tracking_demo.py        # Minimal raw MediaPipe demo (no helper module)\n├── palm_orientation_test.py     # Standalone palm front/back orientation tester\n│\n└── README.md\n```\n\n---\n\n## Configuration\n\nAll three controller files expose the same top-level constants you can tune:\n\n| Constant | Default | Description |\n|---|---|---|\n| `FRAME_R` | `100` px | Margin around the camera frame edge — gestures outside this border are ignored |\n| `SMOOTHENING` | `6` or `7` | Higher = smoother but slower cursor; lower = snappier but jittery |\n\nGesture mappings live in the `FINGER_CONFIG` dict at the top of each file and can be freely remapped.\n\n---\n\n## Installation\n\n**1. Clone the repository**\n```bash\ngit clone https://github.com/Uni-Creator/HandGestureAutomation.git\ncd HandGestureAutomation\n```\n\n**2. Create and activate a virtual environment** *(recommended)*\n```bash\npython -m venv venv\n# Windows\nvenv\\Scripts\\activate\n# macOS / Linux\nsource venv/bin/activate\n```\n\n**3. Install dependencies**\n```bash\npip install opencv-python mediapipe numpy pyautogui pynput \n# Windows only (for the Win32 backend):\npip install pywin32\n```\n\n**4. Run your preferred backend**\n```bash\n# Best on Windows:\npython mouse_control_win32.py\n\n# Cross-platform:\npython mouse_control_pynput.py\npython mouse_control_pyautogui.py\n```\n\nPress **`Q`** or **`Esc`** to quit.\n\n---\n\n## Tips for Best Results\n\n- **Lighting**: Use good, even lighting — avoid strong backlighting behind your hand.\n- **Distance**: Keep your hand roughly **40–70 cm** from the webcam.\n- **Background**: A plain, uncluttered background improves detection reliability.\n- **Orientation**: Keep your palm **facing the camera** (palm front). The orientation guard will suppress input if your hand is turned or flipped.\n- **Active zone**: The **pink/magenta rectangle** shown on screen is the active gesture area. Move your hand within it to control the cursor across the full screen.\n\n---\n\n## Tech Stack\n\n| Library | Purpose |\n|---|---|\n| [MediaPipe](https://mediapipe.dev/) | Real-time hand landmark detection (21 points) |\n| [OpenCV](https://opencv.org/) | Webcam capture and frame rendering |\n| [NumPy](https://numpy.org/) | Coordinate interpolation and clipping |\n| [pynput](https://pynput.readthedocs.io/) | Cross-platform mouse input (pynput backend) |\n| [pyautogui](https://pyautogui.readthedocs.io/) | Cross-platform mouse input (pyautogui backend) |\n| [pywin32](https://github.com/mhammond/pywin32) | Win32 API mouse events (Win32 backend) |\n\n---\n\n## License\n\nThis project is licensed under the **MIT License** — see [LICENSE](LICENSE) for details.\n\n---\n\n## Acknowledgements\n\n- [Google MediaPipe](https://mediapipe.dev/) for the hand tracking model\n- [OpenCV](https://opencv.org/) for computer vision utilities\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funi-creator%2Fhandgestureautomation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funi-creator%2Fhandgestureautomation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funi-creator%2Fhandgestureautomation/lists"}