{"id":27296592,"url":"https://github.com/dirkzon/gesture-recognition","last_synced_at":"2025-04-11T23:43:15.239Z","repository":{"id":171666711,"uuid":"377183597","full_name":"dirkzon/gesture-recognition","owner":"dirkzon","description":"Real time gesture recognition using OpenCV \u0026 Mediapipe.","archived":false,"fork":false,"pushed_at":"2024-12-31T10:40:25.000Z","size":26758,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-12-31T11:31:33.004Z","etag":null,"topics":["artificial-intelligence","gesture-recognition","mediapipe-hands","opencv-python","python","sklearn"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dirkzon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-15T14:03:18.000Z","updated_at":"2024-12-27T08:29:27.000Z","dependencies_parsed_at":null,"dependency_job_id":"89aa6233-b232-4d36-b0a1-a73f699d8b3d","html_url":"https://github.com/dirkzon/gesture-recognition","commit_stats":null,"previous_names":["dirkzon/gesture-recognition"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirkzon%2Fgesture-recognition","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirkzon%2Fgesture-recognition/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirkzon%2Fgesture-recognition/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirkzon%2Fgesture-recognition/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dirkzon","download_url":"https://codeload.github.com/dirkzon/gesture-recognition/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248497902,"owners_count":21113982,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","gesture-recognition","mediapipe-hands","opencv-python","python","sklearn"],"created_at":"2025-04-11T23:43:14.759Z","updated_at":"2025-04-11T23:43:15.220Z","avatar_url":"https://github.com/dirkzon.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gesture recognition\n\n\u003cdiv align=\"center\"\u003e\nReal time gesture recognition using OpenCV \u0026 Mediapipe\n\u003cbr /\u003e\n\u003c/div\u003e\n\u003cdiv align=\"center\"\u003e\n\u003cbr /\u003e\n\n[![license](https://img.shields.io/badge/license-MIT-green)](LICENSE)\n[![made-with-python](https://img.shields.io/badge/Made%20with-Python-1f425f)](https://www.python.org/) \n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./examples/example.gif\" width=\"640\"\u003e\n\u003c/p\u003e\n\n## Contents\n- [About](#about)\n- [Features](#features)\n- [Setup](#setup)\n  - [Prerequisites](#prerequisites)\n  - [Starting the final application](#starting-the-final-application)\n- [Approach](#approach)\n- [Dataset](#dataset)\n- [Work with the data](#work-with-the-data)\n- [Use your own data](#use-your-own-data)\n- [License](#license)\n\n## About\nMade for my fourth semester with Python. The goal of this project was to create a real time gesture recognition app which is able to recognize american sign language gestures. The [model](./model) that this app uses is able to recognize gestures for digits '0' to '9'. The [notebook](Gesture%20recognition.ipynb) explains how this model was made.\n\n## Features\n- __real time:__ gesture recognition in real time.\n- __multiple hands:__ multiple hands can be recognized at once.\n- __ambidextrous:__ recognize gestures on both left and right hand.\n- __debug:__ visualize hand skeleton, confidence \u0026 more.\n\n## Setup\n### Prerequisites\n- Camera/Webcam\n- [Python](https://www.python.org/)\n- [Jupyter](https://jupyter.org/install)\n\n### Starting the final application\n1. CD into the project folder.\n```\ncd gesture-recognition\n```\n2. Create and start a new python environment.\n```\npython -m venv venv\n```\n```\nvenv/Scripts/activate\n```\n3. Install all requirements using the [requirements.txt](requirements.txt) file.\n```\npip install -r requirements.txt\n```\n4. Run the [gesture.py](gesture.py) file.\n```\npython gesture.py\n```\n\n5. Close the application by pressing `q`, and deactivate the enrionment.\n```\ndeactivate\n```\n\n## Approach\nThe [approach](approach.pdf) document explains the score of the project and my approach to different topics like choosing a hand pose estimation model, getting a dataset and training a model. It also contains some reccomentations for all these subjects.\n\n## Dataset\nThe [Sign Language Digits Dataset](https://github.com/ardamavi/Sign-Language-Digits-Dataset) was used to train and test the model.\n\n\u003e*Mavi, A., (2020), “A New Dataset and Proposed Convolutional  Neural Network Architecture for Classification of American Sign Language Digits”, arXiv:2011.08927 [cs.CV]*\n\nThis repository does __NOT__ contain this data. So, if you want to process the images yourself you will have to download the dataset and put all the subfolders into a folder called `images`.\nThe structure should look like this:\n```\n. \n├─ 🗋 Gesture recognition.ipynb \n├─ 🗁 images/\n│  ├─🗀 0/ \n│  ├─🗀 1/ \n│  ├─🗀 2/ \n│  ├─🗀 4/ \n│  ├─🗀 5/ \n│  ├─...\n```\nIf you wish to use your own dataset click [Here](#use-your-own-data) to see how.\n\n## Work with the data\nIf you want to work on pre-processing the dataset yourself you can use the [notebook](Gesture%20recognition.ipynb). It's not necessary to process all the images yourself. the [`dataframes`](dataframes) directory contains both the raw points data and the pre-processed data.\n\nThe [gesture-points-raw](dataframes/gesture-points-raw.csv) data is not cleaned and contains a number of missing values. This dataframe can be used if you want to try your own technique of pre-processing.\n\nThe [gesture-points-processed](dataframes/gesture-points-processed.csv) contains the pre-processed dataframe. This is the pre-processed version of the raw dataframe. All missing values have been fixed, the data has been normalized and a flipped version of the dataframe has been appeded. This dataset can be used if you want to try out your own technique of modeling.\n\n## Use your own data\nThe [notebook](Gesture%20recognition.ipynb) should also work with other datasets of gesture images. But you have to make sure that the images are put into a folder called `images`. The structure should look [the same](#dataset) as if you were using the original dataset. The openpose hand model is __NOT__ included inside this repository. This model can be downloaded [here](https://www.kaggle.com/changethetuneman/openpose-model?select=pose_iter_102000.caffemodel), and needs to be put in the [`openpose`](openpose) directory.\n\nThere are some limitations for the dataset that need to be taken into account:\n\n- The hand in the image must be larger than 60x60 pixels.\n- Lower exposure images work better.\n\nIn the [Dataset restrictions](Dataset%20restrictions.ipynb)  you can see how i found these limitations. In the [approach](approach.pdf) document you can also find some more reccomendations for the dataset.\n\n## License\nThis software is licensed under [MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdirkzon%2Fgesture-recognition","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdirkzon%2Fgesture-recognition","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdirkzon%2Fgesture-recognition/lists"}