{"id":15097645,"url":"https://github.com/ramezze/virtualmouse-handtracking","last_synced_at":"2026-02-15T19:33:55.911Z","repository":{"id":239334874,"uuid":"798870992","full_name":"RamezzE/VirtualMouse-HandTracking","owner":"RamezzE","description":"A python application to control mouse functions using hand gestures leveraging MediaPipe, OpenCV and Tensorflow.","archived":false,"fork":false,"pushed_at":"2024-08-07T15:11:30.000Z","size":39840,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-01T19:46:02.051Z","etag":null,"topics":["computer-vision","handtracking","handtrackingmodule","kivy","machine-learning","mediapipe","mediapipe-hands","mouse","multilayer-perceptron","neural-network","python","scikit-learn","scipy","tensorflow","virtualmouse","webcam","webcam-capture"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RamezzE.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-10T16:40:44.000Z","updated_at":"2024-06-23T10:59:00.000Z","dependencies_parsed_at":"2024-05-14T15:46:41.907Z","dependency_job_id":"5673185e-cbc1-4e4b-ba6c-e225c65fc3a9","html_url":"https://github.com/RamezzE/VirtualMouse-HandTracking","commit_stats":{"total_commits":108,"total_committers":1,"mean_commits":108.0,"dds":0.0,"last_synced_commit":"1601cd4f150efdc3e0ca11a7a0df6b10db01adab"},"previous_names":["ramezze/virtual-mouse---hand-tracking","ramezze/virtualmouse-handtracking"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RamezzE%2FVirtualMouse-HandTracking","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RamezzE%2FVirtualMouse-HandTracking/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RamezzE%2FVirtualMouse-HandTracking/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RamezzE%2FVirtualMouse-HandTracking/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RamezzE","download_url":"https://codeload.github.com/RamezzE/VirtualMouse-HandTracking/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245878930,"owners_count":20687298,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","handtracking","handtrackingmodule","kivy","machine-learning","mediapipe","mediapipe-hands","mouse","multilayer-perceptron","neural-network","python","scikit-learn","scipy","tensorflow","virtualmouse","webcam","webcam-capture"],"created_at":"2024-09-25T16:24:28.771Z","updated_at":"2026-02-15T19:33:55.878Z","avatar_url":"https://github.com/RamezzE.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Virtual Mouse\n\n## Table of Contents\n- [Overview](#overview)\n  - [App Preview](#app-preview)\n  - [Usage Preview](#usage-preview)\n- [Methodology](#methodology)\n- [Settings](#settings)\n- [Prerequisites](#prerequisites)\n- [Building](#building)\n- [Acknowledgements](#acknowledgements)\n- [License](#license)\n\n## Overview\n\nBuilt using OpenCV, MediaPipe, Tensorflow and Kivy for the user interface, Virtual Mouse allows mouse control using several hand gestures mapped to perform different mouse functions such as movement, dragging, clicking, scrolling, zooming etc. These mappings can be changed and configured along with additional settings in runtime through the Kivy application.\n\n### App Preview\n\n\u003cp align=\"left\"\u003e\n\n  \u003cimg src=\"https://github.com/RamezzE/VirtualMouse-HandTracking/assets/117018553/d8aa8a15-9909-4fe9-a4cf-5f111e60f317\" alt=\"Home Screen\" height=\"200\"\u003e\n  \u003cimg src=\"https://github.com/RamezzE/VirtualMouse-HandTracking/assets/117018553/af41b857-a3d2-4b3d-b390-4ddb3d9aa6b7\" alt=\"Camera Feedback Screen\" height=\"200\"\u003e\n  \u003cimg src=\"https://github.com/RamezzE/VirtualMouse-HandTracking/assets/117018553/525fcf55-4758-4024-8715-3205220e4123\" alt=\"Settings Screen 1\" height=\"200\"\u003e\n  \u003cimg src=\"https://github.com/RamezzE/VirtualMouse-HandTracking/assets/117018553/0d81a47c-586b-440e-8f50-29a9a2ee7b35\" alt=\"Settings Screen 2\" height=\"200\"\u003e\n\u003c/p\u003e\n\n### Usage Preview\n\n\u003cdetails\u003e\n\u003csummary\u003eMouse Movement\u003c/summary\u003e\n  \n\u0026nbsp;\n\n  - Absolute Mouse:\n\n  \u003cimg src=\"https://github.com/RamezzE/VirtualMouse-HandTracking/assets/117018553/0ac1098a-0db6-45ee-b873-30e4e710522d\" alt=\"Settings Screen 1\" height=\"300\"\u003e\n\n  - Relative Mouse:\n\n  \u003cimg src=\"https://github.com/RamezzE/VirtualMouse-HandTracking/assets/117018553/305f2263-95d3-4e43-bb4f-eb68835ef1ae\" alt=\"Settings Screen 1\" height=\"300\"\u003e\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eDragging\u003c/summary\u003e\n  \n  \u0026nbsp;\n  \n  \u003cimg src=\"https://github.com/RamezzE/VirtualMouse-HandTracking/assets/117018553/c4654c4d-cc0f-4c72-b2c4-842b509c9696\" alt=\"Settings Screen 1\" height=\"300\"\u003e\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eClicking\u003c/summary\u003e\n  \n  \u0026nbsp;\n  \n  \u003cimg src=\"https://github.com/RamezzE/VirtualMouse-HandTracking/assets/117018553/bcecb601-78ff-4d8a-8528-7a65f930e392\" alt=\"Settings Screen 1\" height=\"300\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eScrolling and Zooming\u003c/summary\u003e\n  \n  \u0026nbsp;\n  \n  \u003cimg src=\"https://github.com/RamezzE/VirtualMouse-HandTracking/assets/117018553/60a7baac-52f6-4a28-8001-d4501033700b\" alt=\"Settings Screen 1\" height=\"300\"\u003e\n\n\u003c/details\u003e\n\n## Methodology\n\nThe Virtual Mouse app follows a structured methodology to achieve accurate hand gesture recognition and mapping to mouse actions for each frame.\n\n![Methodology Flowchart](https://github.com/RamezzE/VirtualMouse-HandTracking/assets/117018553/7db9f201-7720-4c0c-9c5e-944f7876b4dc)\n\nThe dataset used for training the model was manually recorded and is available in training/collected_dataset.zip. \n\n## Settings\n\nA simple guide to the available program's settings.\n\n### Detection Settings\n\n1.  **Detection Confidence**: Sets the minimum confidence percentage for the detection to be considered successful.\n     - **High Detection Confidence**: Ensures that only highly confident hand detections are considered, which significantly reduces false detections but may miss more valid hand detections.\n     - **Low Detection Confidence**: Allows more detections to be considered valid, which can capture more hand movements but may include more false detections.\n\n2. **Tracking Confidence**: Sets the minimum confidence percentage for the hand landmarks to be tracked successfully across frames.\n   - **High Tracking Confidence**: Ensures stable tracking of hand landmarks, which can result in smoother and more accurate gesture recognition but may lose track of hands more easily.\n   - **Low Tracking Confidence**: Allows for more continuous tracking of hand landmarks even with lower confidence, which can maintain tracking better but may introduce some jitter.\n\n3. **Detection Responsiveness**: Adjusts how fast the program reacts to gesture changes. It has four values: Instant, Fast, Normal, and Slow.\n   - **Instant**: Program reacts immediately to gesture changes, providing the quickest response time but may be less stable.\n   - **Fast**: Program reacts quickly to gesture changes, balancing speed and stability.\n   - **Normal**: Program reacts at a moderate speed, providing a stable and responsive experience.\n   - **Slow**: Program reacts more slowly to gesture changes, prioritizing stability over speed.\n\n### Mouse Settings\n\n1. **Toggle Relative Mouse**: Toggle relative mouse mode on or off. This can be toggled using a gesture as well.\n    - **ON**: Introduces touch pad like behavior, moving the mouse relative to its previous position and set sensitivity. This works using multiple screens.\n    - **OFF**: Maps hand position in camera to screen position depending on screen size. This does not work using more than one screen.\n\n2. **Relative Mouse Sensitivity**: Sets the mouse sensitivity to use when relative mouse mode is on.\n\n3. **Scroll Sensitivity**: Sets the sensitivity or speed of scrolling when using the set gesture.\n\n### Gesture Mappings\n\nYou can customize which gestures perform specific mouse actions (e.g., left click, right click, scroll, idle) through the settings in the Kivy application. This enables you to tailor the control scheme to your needs.\n\nTo modify gesture mappings:\n1. Open the application and navigate to the settings screen.\n2. Select the \"Gesture Settings\" option.\n3. Change the mouse actions for the desired gestures.\n\n## Prerequisites\n- Python\n- Pip\n\n### Linux\n- You can run these commands on linux to install python, pip, python-venv and other dependencies\n\n```\nsudo apt-get update\nsudo apt install python3\nsudo apt install python3-pip\nsudo apt install python3-venv\nsudo apt-get install python3-tk python3-dev\n```\n\n### Windows\n- Install python from the official website\n- Check if pip is already installed by running: `pip help`\n- If pip is not installed, please check this [guide](https://www.geeksforgeeks.org/download-and-install-pip-latest-version/#windows) for installing pip on Windows\n\n## Building\n\n1. **Clone the repository**\n\n    ```\n    git clone https://github.com/RamezzE/VirtualMouse-HandTracking.git\n    ```\n\n2. **Navigate to project folder**\n  \n    ```\n    cd VirtualMouse-HandTracking\n    ```\n\n3. **Create and activate a python virtual environment**\n\n      - Linux\n      \n          ```\n          python3 -m venv venv\n          source venv/bin/activate\n          ```\n      \n      - Windows\n      \n          ```\n          python -m venv venv\n          venv\\Scripts\\activate\n          ```\n\n4. **Install necessary pip packages**\n\n    ```\n    pip install -r requirements.txt\n    ``` \n\n    - If the above command does not work or throws an error, run the below command instead\n    \n        ```\n        pip install numpy mediapipe scikit-learn kivy[base] mouse pyautogui pyaml opencv-python\n        ```\n    \n    - If you want to run training.ipynb, then install these extra packages as well\n    \n        ```\n        pip install ipykernel tensorflow pandas xgboost\n        ```\n\n5. **Run main application file**\n\n    ```\n    python main.py\n    ```\n\n- Optionally, if you'd like to run the script directly without running the Kivy application, you can run the alternative main file\n\n    ```\n    python main_no_gui.py\n    ```\n\n## Acknowledgements\n\nMost of the icons used are provided by [Icons8](https://icons8.com/)\n\n## License\n\nThis project is licensed under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Framezze%2Fvirtualmouse-handtracking","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Framezze%2Fvirtualmouse-handtracking","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Framezze%2Fvirtualmouse-handtracking/lists"}