{"id":35159350,"url":"https://github.com/ansh2222949/ai-mouse","last_synced_at":"2025-12-29T18:04:47.404Z","repository":{"id":330568212,"uuid":"1122608341","full_name":"ansh2222949/ai-mouse","owner":"ansh2222949","description":"Real-time hand gesture controlled mouse using computer vision and hybrid ML","archived":false,"fork":false,"pushed_at":"2025-12-25T18:05:45.000Z","size":12,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-27T23:18:54.061Z","etag":null,"topics":["computer-vision-opencv","gesture-recognition","hand-tracking","machine-learning","mediapipe","python","real-time-systems","research-project"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ansh2222949.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-25T05:50:09.000Z","updated_at":"2025-12-27T13:41:57.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ansh2222949/ai-mouse","commit_stats":null,"previous_names":["ansh2222949/ai-mouse"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/ansh2222949/ai-mouse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ansh2222949%2Fai-mouse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ansh2222949%2Fai-mouse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ansh2222949%2Fai-mouse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ansh2222949%2Fai-mouse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ansh2222949","download_url":"https://codeload.github.com/ansh2222949/ai-mouse/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ansh2222949%2Fai-mouse/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28119514,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-29T02:00:07.021Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision-opencv","gesture-recognition","hand-tracking","machine-learning","mediapipe","python","real-time-systems","research-project"],"created_at":"2025-12-28T17:52:26.920Z","updated_at":"2025-12-29T18:04:47.398Z","avatar_url":"https://github.com/ansh2222949.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003ch1\u003e🖱️ AI Mouse\u003c/h1\u003e\n  \u003ch3\u003eReal-Time Hand Gesture Control System\u003c/h3\u003e\n  \u003cp\u003e\n    \u003cb\u003eComputer Vision • Hybrid Machine Learning • Low Latency\u003c/b\u003e\n  \u003c/p\u003e\n  \n  \u003cp\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Python-3.10+-blue?style=for-the-badge\u0026logo=python\u0026logoColor=white\" alt=\"Python\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/ML-Hybrid%20(KNN%2BForest)-orange?style=for-the-badge\" alt=\"Machine Learning\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Vision-MediaPipe-green?style=for-the-badge\u0026logo=opencv\" alt=\"Computer Vision\"\u003e\n  \u003c/p\u003e\n\u003c/div\u003e\n\n---\n\n## 🧠 Project Overview\n\n**AI Mouse** is a real-time computer vision research project that explores mouse control using hand gestures captured via a standard webcam.\n\nUnlike typical deep learning approaches, this system combines **MediaPipe** hand tracking with a **Hybrid Machine Learning architecture (KNN + Random Forest)**. This specific design choice ensures ultra-low latency, stability, and the ability to adapt to new gestures on the fly without heavy retraining.\n\n\u003e 🎯 **Focus:** System design, real-time performance, and practical ML decision-making rather than deep learning complexity.\n\n---\n\n## ✨ Key Features\n\n* **🎥 Real-Time Tracking:** Uses MediaPipe for robust hand landmark detection.\n* **✋ Gesture Control:** Full mouse navigation including **Move, Scroll, and Click**.\n* **🧠 Hybrid ML Engine:**\n    * **KNN:** For fast, online incremental adaptation.\n    * **Random Forest:** For stabilizing confidence scores.\n* **🎯 Smart Execution:** Temporal buffering to reduce jitter and false positives.\n* **📦 Modular Architecture:** Clean separation between vision, logic, and execution layers.\n* **🖥️ Fully Offline:** No internet connection required.\n\n---\n\n## 🧱 Technical Architecture\n\nThe system avoids deep learning to prioritize speed and interpretability.\n\n```text\nAI_MOUSE/\n│\n├── core/\n│   ├── config.py        # System sensitivity \u0026 configuration\n│   ├── features.py      # Hand landmark feature extraction\n│   ├── model.py         # Hybrid KNN + Random Forest logic\n│   ├── actions.py       # PyAutoGUI execution (Mouse/Click)\n│   └── __init__.py\n│\n├── main.py              # Camera loop \u0026 orchestration\n├── requirements.txt     # Dependencies\n└── README.md            # Documentation\n\n```\n\n### 🧠 Why Hybrid ML? (The Research Angle)\n\nThis problem demands **ultra-low latency** and **online learning**. Deep learning models often introduce unnecessary overhead.\n\n1. **KNN (K-Nearest Neighbors):** Allows for instant adaptation to a specific user's hand shape.\n2. **Random Forest:** Acts as a stabilizer to filter out noise from the webcam.\n3. **Result:** A system that is faster and more responsive than heavy neural networks for this specific task.\n\n---\n\n## 🧪 Gestures \u0026 Controls\n\nGestures are trained **live** during runtime to match the user's specific hand.\n\n| ID | Gesture Name | Action |\n| --- | --- | --- |\n| **1** | **MOVE** | Cursor follows hand movement |\n| **2** | **SCROLL** | Scroll Up / Down |\n| **3** | **CLICK** | Left Mouse Click |\n| **4** | **IDLE** | No Action (Safety state) |\n\n### ⌨️ Keyboard Controls\n\n| Key | Function |\n| --- | --- |\n| `1` / `2` / `3` / `4` | **Train** the respective gesture (Hold to capture data) |\n| `s` | **Save** trained model data locally |\n| `r` | **Reset** / Clear current calibration |\n| `ESC` | **Exit** the program |\n\n---\n\n## 🚀 How to Run\n\n### 1️⃣ Prerequisites\n\n* Python 3.10+\n* A working Webcam\n\n### 2️⃣ Installation\n\n```bash\n# Create Virtual Environment\npy -3.10 -m venv .venv\n.venv\\Scripts\\activate\n\n# Install Dependencies\npip install -r requirements.txt\n\n```\n\n### 3️⃣ Execution\n\n```bash\npython main.py\n\n```\n\n---\n\n## ⚠️ Safety \u0026 Warning\n\nThis project creates a virtual mouse interface. To ensure smooth movement, the failsafe is disabled:\n`pyautogui.FAILSAFE = False`\n\n**If the mouse behaves unexpectedly or gets stuck:**\n\n1. Press **`ESC`** immediately to kill the script.\n2. Or press **`Alt + Tab`** to switch windows.\n3. Or close the **OpenCV window**.\n\n\u003e *Use with caution. This behavior is intentional for experimentation.*\n\n---\n\n## 📌 Notes\n\n* **User Specific:** Trained data (`.pkl`) is specific to your hand and lighting conditions. It is not synced to Git.\n* **OS:** Designed and tested on **Windows**.\n* **Scope:** This is an experimental research project, not intended for production accessibility tools.\n\n---\n\n## 🔮 Future Improvements\n\n* [ ] Visual overlays for gesture confidence.\n* [ ] Dynamic sensitivity tuning via GUI.\n* [ ] Comparative latency study against CNN models.\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\u003cb\u003eShared for Educational \u0026 Research Purposes\u003c/b\u003e\n\u003c/div\u003e\n\n```\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fansh2222949%2Fai-mouse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fansh2222949%2Fai-mouse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fansh2222949%2Fai-mouse/lists"}