{"id":19888461,"url":"https://github.com/dynamicanupam/hand_gesture_recognition_using_cnn-rnn","last_synced_at":"2025-10-13T12:19:40.151Z","repository":{"id":236896451,"uuid":"793371785","full_name":"dynamicanupam/Hand_Gesture_Recognition_using_CNN-RNN","owner":"dynamicanupam","description":"Build deep learning model for detecting hand gestures for Smart TVs using CNN and RNN","archived":false,"fork":false,"pushed_at":"2024-09-29T09:07:24.000Z","size":350,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-01T04:42:07.486Z","etag":null,"topics":["3d-convs","cnn-rnn","generator-functions","model-building-and-training","model-evaluation","model-tuning"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dynamicanupam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-29T05:18:46.000Z","updated_at":"2024-09-29T09:09:14.000Z","dependencies_parsed_at":"2024-05-06T16:47:42.306Z","dependency_job_id":"0cd9712f-4d5e-4e9c-851b-4adff35b1c8f","html_url":"https://github.com/dynamicanupam/Hand_Gesture_Recognition_using_CNN-RNN","commit_stats":null,"previous_names":["dynamicanupam/hand-gesture-recognition","dynamicanupam/hand_gesture_recognition_using_cnn-rnn"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dynamicanupam%2FHand_Gesture_Recognition_using_CNN-RNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dynamicanupam%2FHand_Gesture_Recognition_using_CNN-RNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dynamicanupam%2FHand_Gesture_Recognition_using_CNN-RNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dynamicanupam%2FHand_Gesture_Recognition_using_CNN-RNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dynamicanupam","download_url":"https://codeload.github.com/dynamicanupam/Hand_Gesture_Recognition_using_CNN-RNN/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241317596,"owners_count":19943201,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-convs","cnn-rnn","generator-functions","model-building-and-training","model-evaluation","model-tuning"],"created_at":"2024-11-12T18:07:19.318Z","updated_at":"2025-10-13T12:19:40.084Z","avatar_url":"https://github.com/dynamicanupam.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Gesture Recognition\n\u003e Make a Smart TV system which can control the TV with user’s hand gestures as the remote control.\n---\n\n## Problem Statement\nImagine you are working as a data scientist at a home electronics company which manufactures state of the art smart televisions. You want to develop a cool feature in the smart-TV that can recognise five different gestures performed by the user which will help users control the TV without using a remote.\n\nThe gestures are continuously monitored by the webcam mounted on the TV. Each gesture corresponds to a specific command:\n\nThumbs up:  Increase the volume\n\nThumbs down: Decrease the volume\n\nLeft swipe: 'Jump' backwards 10 seconds\n\nRight swipe: 'Jump' forward 10 seconds  \n\nStop: Pause the movie\n\n## Understanding the Dataset\nThe training data consists of a few hundred videos categorised into one of the five classes. Each video (typically 2-3 seconds long) is divided into a sequence of 30 frames(images). These videos have been recorded by various people performing one of the five gestures in front of a webcam - similar to what the smart TV will use.\n\nThe data is in a zip file. The zip file contains a 'train' and a 'val' folder with two CSV files for the two folders. \n\nThese folders are in turn divided into subfolders where each subfolder represents a video of a particular gesture. Each subfolder, i.e. a video, contains 30 frames (or images). Note that all images in a particular video subfolder have the same dimensions but different videos may have different dimensions. Specifically, videos have two types of dimensions - either 360x360 or 120x160 (depending on the webcam used to record the videos). Hence, you will need to do some pre-processing to standardise the videos. \n\n## Deep Learning Models used\n1. Conv3D\n2. CNN with LSTM\n3. CNN with GRU\n4. CNN with GRU (with trainable weights of Transfer Learning)\n\n## Conclusion\nBased on the results CNN with GRU (with trainable weights of Transfer Learning) model\nperforming well on the dataset provided for Gesture Recognition.\nSelected model: **CNN with GRU (with trainable weights of Transfer Learning)**\n- Training Accuracy: 0.99\n- Validation Accuracy: 0.99\n- Batch Size: 5\n- Frames: 16\n- Number of epochs: 20\n- Model File Name: model-00020-0.01408-0.99698-0.02186-1.00000.keras\n- Model Loss and Accuracy comparison b/w Train and Validation set:\n  ![image](https://github.com/dynamicanupam/Hand-Gesture-Recognition/assets/61014822/8c92e863-bd17-4635-82d0-6c18eaa6a3e6)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdynamicanupam%2Fhand_gesture_recognition_using_cnn-rnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdynamicanupam%2Fhand_gesture_recognition_using_cnn-rnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdynamicanupam%2Fhand_gesture_recognition_using_cnn-rnn/lists"}