{"id":19671427,"url":"https://github.com/lasithaamarasinghe/hand-gesture-math-solver","last_synced_at":"2025-09-04T07:35:29.632Z","repository":{"id":244814740,"uuid":"815971430","full_name":"LasithaAmarasinghe/Hand-Gesture-Math-Solver","owner":"LasithaAmarasinghe","description":"This project demonstrates a real-time hand gesture recognition system using Python, OpenCV, and Gemini AI by Google.","archived":false,"fork":false,"pushed_at":"2024-06-17T05:11:19.000Z","size":4721,"stargazers_count":5,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-05T20:52:15.631Z","etag":null,"topics":["computer-vision","cvzone","gemini-ai","hand-recognition","numpy","opencv","pillow","python","visual-studio-code"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LasithaAmarasinghe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"MIT-LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-16T17:31:13.000Z","updated_at":"2025-02-25T00:53:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"76929002-afab-4f8d-86e1-541ad9100afc","html_url":"https://github.com/LasithaAmarasinghe/Hand-Gesture-Math-Solver","commit_stats":null,"previous_names":["lasithaamarasinghe/hand-gesture-math-solver"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/LasithaAmarasinghe/Hand-Gesture-Math-Solver","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LasithaAmarasinghe%2FHand-Gesture-Math-Solver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LasithaAmarasinghe%2FHand-Gesture-Math-Solver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LasithaAmarasinghe%2FHand-Gesture-Math-Solver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LasithaAmarasinghe%2FHand-Gesture-Math-Solver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LasithaAmarasinghe","download_url":"https://codeload.github.com/LasithaAmarasinghe/Hand-Gesture-Math-Solver/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LasithaAmarasinghe%2FHand-Gesture-Math-Solver/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273573595,"owners_count":25129881,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-04T02:00:08.968Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","cvzone","gemini-ai","hand-recognition","numpy","opencv","pillow","python","visual-studio-code"],"created_at":"2024-11-11T17:08:45.871Z","updated_at":"2025-09-04T07:35:29.612Z","avatar_url":"https://github.com/LasithaAmarasinghe.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hand-Gesture-Math-Solver\n\n## Overview\n- This project demonstrates a real-time hand gesture recognition system using Python, OpenCV, and Gemini AI by Google.\n- The system allows users to solve math problems by drawing them in the air.\n- The repository contains all the code and resources needed to implement this project.\n\n## Features\n\n- Real-time hand detection and tracking using OpenCV and cvzone.\n- Recognition of specific hand gestures to perform actions like drawing, clearing canvas, and requesting math problem solutions.\n- Integration with Gemini AI for solving math problems based on gestures.\n\n## Technologies/Tools\n\n* Python 3.10.12\n* cvzone `pip install cvzone`\n* OpenCV `pip install opencv-python`  \n* Gemini AI `pip install google-generativeai`\n* Python packages\n  * numpy `pip install numpy` \n  * pillow `pip install pillow`\n \n![Python](https://img.shields.io/badge/python-3670A0?logo=python\u0026logoColor=FFFF00)\n![cvzone](https://img.shields.io/badge/cvzone_-orange)\n![OpenCV](https://img.shields.io/badge/opencv-%23white.svg?logo=opencv\u0026logoColor=white)\n![Gemini AI](https://img.shields.io/badge/Gemini%20AI_-%238E75B2?logo=googlegemini\u0026logoColor=white)\n![NumPy](https://img.shields.io/badge/numpy-%23013243.svg?logo=numpy\u0026logoColor=white)\n![pillow](https://img.shields.io/badge/pillow_-blue)\n\n* C++ compiler - install [Visual Studio](https://visualstudio.microsoft.com/downloads/)\n\n## Installation\n \n  1. **Clone the repository:**\n \n     ```bash\n     git clone https://github.com/LasithaAmarasinghe/Hand-Gesture-Math-Solver.git\n     cd Hand-Gesture-Math-Solver\n     ```\n \n  2. **Create and activate a virtual environment (optional but recommended):**\n\n     ```bash\n     python -m venv venv\n     source venv/bin/activate   # On Windows use `venv\\Scripts\\activate`\n     ```\n \n  3. **Install the required packages:**\n \n     ```bash\n     pip install -r requirements.txt\n     ```\n## Usage\n\n  1. **Get a Gemini API key**\n\n     You can get an API key from [here](https://aistudio.google.com/app/apikey)\n\n  2. **Replace API key in the code with the obtained API key**\n     \n  3. **Execute the main script**\n\n```bash\npython math_solver.py\n```\n 4. **Interact with the Application**\n   \n     * **Live Feed**\n        * The application will display the live feed from your webcam.\n     * **Perform Hand Gestures**\n        * Draw: Use one finger to draw on the canvas.\n        * Clear: Use one finger to clear the canvas.\n        * Solve Math Problem: Use all fingers except the thumb to submit a math problem for solving.\n      \n## Code Explanation\n\n  1. **getHandInfo(img)**\n\n     * **Purpose:** Detects and tracks the user's hand in each frame of the webcam feed.\n     * **Implementation:** Uses the HandTrackingModule to identify and monitor hand movements.\n\n  2. **draw(info, prev_pos, canvas)**\n\n     * **Purpose:** Draws on the canvas based on the detected hand gestures.\n     * **Parameters:**\n          * `info`: Information about the detected hand gestures.\n          * `prev_pos`: Previous position of the drawing.\n          * `canvas`: Canvas to draw on.\n\n  3. **sendToAI(model, canvas, fingers)**\n\n     * **Purpose:** Sends the drawn content to Google's Generative AI for math problem solving based on recognized gestures.\n     * **Parameters:**\n          * `model`: Instance of the Generative AI model.\n          * `canvas`: Canvas containing the drawn content.\n          * `fingers`: Array indicating the state of fingers (e.g., for gesture recognition).\n\n\n\n## Hand Detection Concept\n\n\u003cimg src=\"https://github.com/LasithaAmarasinghe/Hand-Gesture-Math-Solver/assets/106037441/196a14ec-6067-494c-8e06-69873fa418f3\" \u003e\n\n___\n\n**Left-Right Hand Detection**\n\n\u003cimg src=\"https://github.com/LasithaAmarasinghe/Hand-Gesture-Math-Solver/assets/106037441/c0eaf562-bb89-4b47-b9f8-4bb96ec3e442\" width=\"400\" height=\"240\"\u003e\n\u003cimg src=\"https://github.com/LasithaAmarasinghe/Hand-Gesture-Math-Solver/assets/106037441/a7c1a542-8b85-42a2-87fa-bb20ff280a1e\" width=\"400\" height=\"240\"\u003e\n\n___\n\n**Hand Gesture Detection**\n\n\u003cimg src=\"https://github.com/LasithaAmarasinghe/Hand-Gesture-Math-Solver/assets/106037441/56434be0-8338-41e2-b060-5b14e69f6325\" width=\"400\" height=\"280\"\u003e\n\u003cimg src=\"https://github.com/LasithaAmarasinghe/Hand-Gesture-Math-Solver/assets/106037441/b66a3d35-f7f7-4b75-b7db-c93ee40d6841\" width=\"400\" height=\"280\"\u003e\n\n## Results\n\n**Math Expression Detection \u0026 Canvas** \n\n\u003cimg src=\"https://github.com/LasithaAmarasinghe/Hand-Gesture-Math-Solver/assets/106037441/e9558800-a17f-432d-a4f5-d00d0a352cfb\" width=\"400\" height=\"240\"\u003e\n\u003cimg src=\"https://github.com/LasithaAmarasinghe/Hand-Gesture-Math-Solver/assets/106037441/64664179-a2cc-43a2-af2b-409a6beb4325\" width=\"400\" height=\"240\"\u003e\n\n## License\n \n * This project is licensed under the MIT License. See the [LICENSE](MIT-LICENSE.txt) file for details.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flasithaamarasinghe%2Fhand-gesture-math-solver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flasithaamarasinghe%2Fhand-gesture-math-solver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flasithaamarasinghe%2Fhand-gesture-math-solver/lists"}