Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/bottomsnode/sct_ml_4

This project develops a hand gesture recognition model to control computers through gestures. Using deep learning and computer vision, the model identifies hand gestures in real-time and triggers corresponding actions, offering a touchless and intuitive interaction method for digital applications.
https://github.com/bottomsnode/sct_ml_4

computervision datascience-machinelearning handgesture-recognition real-time-processing

Last synced: 2 months ago
JSON representation

This project develops a hand gesture recognition model to control computers through gestures. Using deep learning and computer vision, the model identifies hand gestures in real-time and triggers corresponding actions, offering a touchless and intuitive interaction method for digital applications.

Awesome Lists containing this project

README 

        


  
Hand Gesture Recognition


  


    

    

    

    


    Building a hand gesture recognition model and using it to identify hand gestures in real-time to trigger actions on a computer

  





  
Table of Contents


  
    

        
  1. 

          About the Project

          

        
    2. 

        
  2. 

          Prerequisites

        
    3. 

        
  3. 

          Setup

        
    4. 

        
  4. 

          Acknowledgments

        
    5. 

      



About the Project





  The COVID-19 pandemic has inevitably accelerated the adoption of a number of contactless Human-Computer Interaction (HCI) technologies, one of which is the hand gesture control technology. Hand gesture-controlled applications are widely used across various industries, including healthcare, food services, entertainment, smartphone, and automotive.



  In this project, a hand gesture recognition model is trained to recognize static and dynamic hand gestures. The model is used to predict hand gestures in real-time through the webcam. Depending on the hand gestures predicted, the corresponding keystrokes (keyboard shortcuts) will be sent to trigger actions on a computer.




Built with



* [Keras](https://keras.io/)

* [OpenCV](https://opencv.org/)

* [Plotly](https://plotly.com/)

* [pynput](https://pynput.readthedocs.io/en/latest/)

* [keras-hypetune](https://github.com/cerlymarco/keras-hypetune)



Dataset





  The dataset used is a subset of the LeapGestRecog dataset from

  Kaggle. It is a large collection of labeled images of humans performing hand gestures in front of a camera.

  

  In this project, 10 classes of hand gestures have been selected to train the hand gesture recognition model.




Example Usage



Any actions on a computer can be triggered as long as they are linked to a keyboard shortcut. For simplicity, this project is configured to trigger actions on YouTube because it has its own built-in keyboard shortcuts.



The table below shows the hand gestures and the actions they trigger on YouTube.





  

    

      Hand gesture

      Action

    

    

      Swiping Left

        
swiping_left

      Fast forward 10 seconds

    

    

      Swiping Right

      
swiping_right

      Rewind 10 seconds

    

    

      Swiping Down

      
swiping_down

      Previous video

    

    

      Swiping Up

      
swiping_up

      Next video

    

    

      Sliding Two Fingers Down

      
sliding_two_fingers_down

      Decrease volume

    

    

      Sliding Two Fingers Up

      
sliding_two_fingers_up

      Increase volume

    

    

      Thumb Down
thumb_down

      Mute / unmute

    

    

      Thumb Up
thumb_up

      Enter / exit full screen

    

    

      Stop Sign
stop_sign

      Play / Pause

    

    

      No Gesture
no_gesture

      No action

    

  




Project Outline



    

      
  1. 

        Data Exploration

        
      

            
    • Explore class distribution of training and validation data.

            
      Training data:class_distribution_train

            
      Validation data:class_distribution_validation

      • 

          
    

      
    2. 

      
  2. 

        Data Extraction 

        
      

            
    • Extract training and validation data of the selected classes from the dataset.
      • 

          
    

      
    3. 

      
  3. 

        Hyperparameter Tuning 

        
      

            
    • Perform grid search to determine the optimal values for dropout and learning rate.
      • 

          
    

      
    4. 

      
  4. 

        Model Training 

        
      

            
    • Build a 3D ResNet-101 model with the optimal hyperparameters.
      • 

            
    • Compile the model.
      • 

            
    • Train the model.
      • 

          
    

      
    5. 

      
  5. 

        Classification 

        
      

            
    • Read frames from the webcam, predict the hand gestures in the frames using the model, and send the corresponding keystrokes to trigger actions on the computer.
      • 

          
    

      
    6. 




Prerequisites



* Python 3.7.9 or above



Setup



  ```sh

  pip install -r requirements.txt

```



Acknowledgments



This documentation should now reflect the specifics of your project and give proper credit to you as the developer. Let me know if you need any further adjustments!