{"id":20909472,"url":"https://github.com/jai0212/ai-virtual-painter","last_synced_at":"2026-04-05T21:03:06.475Z","repository":{"id":226368631,"uuid":"768497503","full_name":"Jai0212/AI-Virtual-Painter","owner":"Jai0212","description":"Real time finger and object painting using advanced computer vision technology with custom hand gestures and customizable colours","archived":false,"fork":false,"pushed_at":"2024-03-07T18:30:53.000Z","size":15,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-02T03:37:49.294Z","etag":null,"topics":["artificial-intelligence","cmake","computer-vision","cpp","mediapipe","numpy","opencv","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Jai0212.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-07T07:36:53.000Z","updated_at":"2024-03-07T07:52:23.000Z","dependencies_parsed_at":"2024-11-18T14:11:31.838Z","dependency_job_id":"ccd728ae-eee3-47f4-b151-de1894ded0e7","html_url":"https://github.com/Jai0212/AI-Virtual-Painter","commit_stats":null,"previous_names":["jai0212/ai-virtual-painter"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Jai0212/AI-Virtual-Painter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jai0212%2FAI-Virtual-Painter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jai0212%2FAI-Virtual-Painter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jai0212%2FAI-Virtual-Painter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jai0212%2FAI-Virtual-Painter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Jai0212","download_url":"https://codeload.github.com/Jai0212/AI-Virtual-Painter/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jai0212%2FAI-Virtual-Painter/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268695182,"owners_count":24291983,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-04T02:00:09.867Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","cmake","computer-vision","cpp","mediapipe","numpy","opencv","python"],"created_at":"2024-11-18T14:11:21.204Z","updated_at":"2025-12-30T21:50:23.900Z","avatar_url":"https://github.com/Jai0212.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Virtual Painter\n\nReal-time finger and object painting using advanced computer vision technology with custom hand gestures and customizable colours.\nThe project has 2 parts:\n\n### Finger Painter\nUses hand detection models/algorithms to let you paint in the real \nworld in real-time using your finger. Allows you to change the colour of the ink using hand gestures (thumbs up \nand down) and allows you to erase the text using your fist.\nImplemented in Python and uses OpenCV, MediaPipe and NumPy.\n\n### Chroma Painter\nUses object and colour detection models/algorithms to allow you to write in the real world in real-time using an object \nof a particular colour. The colour of the object is the colour of the text.\nImplemented in C++ and uses OpenCV.\n\n\u003cimg src=\"https://github.com/Jai0212/AI-Virtual-Painter/assets/86296165/4e552f5c-e403-4f1c-b140-a014d1478a5c\" alt=\"Screenshot\" width=\"1000\"\u003e\nFinger Painter showing the hand tracking and the various colours\n\n## Features\n- **Real-Time Painting using your Finger:** Allows you to paint virtually in the real world with almost no latency using\nyour finger\n- **Custom Gestures:** Seamlessly change ink color with intuitive gestures like thumbs up and thumbs down\n- **Erasing:** Erase content effortlessly by making a fist gesture\n- **Free Movement:** Utilize multi-finger gestures to move around without leaving a trace\n- **Painting with Real-World Objects:** Paint with the color of a real-world coloured object, such as a red pen\n- **Accurate Real-Time Display:** Real-time updates and precise gesture/object detection for accurate painting on the screen\n- **Advanced Detection/Tracking Models:** Accuracy enhanced with specially optimized hand and object detection models/algorithms\n- **User-Friendly:** Intuitive gestures with an easy-to-use intuitive interface\n- **Expandable:** Ability to expand the project by adding more colours and gestures based on personal preferences\n\n\n![Screenshot 2024-03-07 at 2 41 44 AM](https://github.com/Jai0212/AI-Virtual-Painter/assets/86296165/d8b75c81-f2fc-4ba2-906a-dd0cebb48415)\nFinger Painter showing the erase option when doing the fist gesture\n\n## Technical Aspects\nThe program was created in CLion using C++ and Python. The libraries used were OpenCV, MediaPipe and NumPy.\n\n### Finger Painter\nThe computer vision aspect of the project was implemented using OpenCV.\nThe hand-tracking algorithm was implemented using MediaPipe. The custom gestures\nfor changing colours and erasing were created from scratch by analysis of the hand \ncoordinates.\n\nThere are a total of 4 colours in the finger virtual painter (more colours can be added \nby adding the RGB values inside the list). The initial colour is purple. When you do thumbs up or down,\nthe colour changes. If you indicate a fist, the eraser will toggle. This is done by checking the\nparticular distance and arrangement of the hand coordinates at each frame for accuracy. \nIn case you want to move your hand around (without drawing), you can have more than one finger up.  \n\nAll this was accomplished by creating specific functions and analyzing the hand coordinates.\n\n### Chroma Painter\nThe object detection was achieved with the help of OpenCV.\nI first created a colour and object detection model and configured the colours to enable the colour \ndetection model to identify the particular colour.\n\nIf found, the colour model matches the colour with the set of available colours. With the help of\ncomputer vision object detection, the model detects the object and then draws the particular colour on\nthe topmost part of the object. This allows you to draw with almost any colour in the real world with the\nhelp of any real object.\n\n### General Note\nThe webcam image refreshes with every change in movement thereby providing accurate and real time updates based\non the gesture/object movement.\n\nThe drawing was imprinted in the real world by inverting the images and performing specific operations\nto allow real-time and real-world display of your drawings as if they were floating in the air.\n\nI have left the hand tracking lines and the object detector indicator on for a better understanding of how\nthe project works.\n\n![Screenshot 2024-03-07 at 2 42 25 AM](https://github.com/Jai0212/AI-Virtual-Painter/assets/86296165/484ec2df-46cb-4e20-bce0-612e27a7fd07)\nChroma Painter showing the object detection and drawing based on colour of object\n\n## Usage and Requirements\nThe program can be run on any normal laptop with a webcam. To use the project, you can use any IDE that \nsupports C++ (3.17) and Python (3.11). You need to download all the files as is.\n\n### Finger Painter\nTo use this, you need Python and any IDE that runs Python. You will first need to download the packages\nopencv-python, mediapipe and numpy. You can do so using the pip install command in the terminal or \ndirectly from the IDE.\n\nTo run the program, in the directory 'Finger_Painter', run the file 'FingerPainter'. The webcam\nwill open up and you can use the project. Raise your index finger to draw. Close your fist and move it \naround to erase. Make a thumbs up or thumbs down to change the colour. Hold up more than one finger \nto move around the webcam region without drawing.\n\n### Chroma Painter\nTo use this, you will need C++ and any IDE that runs C++. You will first need to install opencv using\nHomebrew (use any online tutorial to learn how to install opencv using Homebrew). You might need to \nchange some file locations depending on where your opencv package is saved in the CMakeLists.txt file.\n\nTo run the program, run the main.cpp file and the webcam should open up. Take any coloured object, for\nexample, a red pen/cup/marker or any holdable red item. Using that, you will be able to draw.\n\n### General Note\n\nFor better usage of the program, I would recommend standing about 1 meter away from your webcam for\nproper hand detection and more screen space to draw. Make sure only one hand is visible to the webcam. \nTo stop, you must terminate the program from the IDE.\nIn case the webcam does not open up when you run the file, try changing the '1' in the line of code\n'web_cam = cv.VideoCapture(1)' to 0 or 2. It depends on your laptop and which webcam you want to use.\n\n## Acknowledgement\nI worked on this project alone and will not be actively working on the project anymore \n(I will be creating other related projects). However, I would love any suggestions/feedback/collaborative requests.\n\n## Author and Date\nby Jai Joshi  \n7th March, 2024\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjai0212%2Fai-virtual-painter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjai0212%2Fai-virtual-painter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjai0212%2Fai-virtual-painter/lists"}