https://github.com/yashas-hm/asl-gesture-sensing

ML Model that translates Hand Motion Gestures to textual data
https://github.com/yashas-hm/asl-gesture-sensing

machine-learning mediapipe-hands mediapipe-pose python tensorflow

Last synced: 3 months ago
JSON representation

ML Model that translates Hand Motion Gestures to textual data

• Host: GitHub
• URL: https://github.com/yashas-hm/asl-gesture-sensing
• Owner: yashas-hm
• License: mit
• Created: 2022-02-28T17:02:53.000Z (over 4 years ago)
• Default Branch: main
• Last Pushed: 2024-02-24T19:02:52.000Z (over 2 years ago)
• Last Synced: 2024-02-24T20:23:44.680Z (over 2 years ago)
• Topics: machine-learning, mediapipe-hands, mediapipe-pose, python, tensorflow
• Language: Jupyter Notebook
• Homepage:
• Size: 25.6 MB
• Stars: 0
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# 🤚 ASL Gesture Sensing Device (Harpocrates)

Welcome to **Harpocrates**, an innovative device designed to detect American Sign Language (ASL) gestures and provide real-time predictions. Currently, it supports basic signs like "Hello" and "Thank you," with the potential for expanding its repertoire through further training.

## 🔧 Technologies Used

`Python` `TensorFlow` `MediaPipe`

## 📝 Project Description

**Harpocrates** employs MediaPipe for hand, pose, and face detection, utilizing a Sequential Model with LSTM and Dense Layers for prediction. It captures 30 frames per action, ensuring accurate recognition. Once a gesture is predicted, it is converted to speech, making communication seamless.

**Harpocrates** serves as a bridge between sign language users and those unfamiliar with it, facilitating communication and understanding.

## 📊 Categorical Accuracy Graph

![Categorical Accuracy Graph](Media/accuracy.png)

This graph illustrates the categorical accuracy's progression with increasing epochs during model training.

## 🔄 Confusion Matrix for Predictions

![Confusion Matrix](Media/confusion_matrix.png)

## 📸 Screenshots

![Face, Pose, and Hands Detection](Media/Screenshot%20(10).png)

Face, pose, and hands detection in action.

![Real-Time Gesture Prediction](Media/detector.gif)

Real-time gesture prediction.

![Model Image 1](Media/1.png)

![Model Image 2](Media/2.png)

3D model of **Harpocrates**.

## 🔗 References

This project draws inspiration and guidance from [Nicholas Renotte's tutorial](https://www.youtube.com/watch?v=doDUihpj6ro).

## 📜 License

[![License](https://img.shields.io/github/license/Ileriayo/markdown-badges?style=for-the-badge)](./LICENSE)

Feel free to explore and contribute to **Harpocrates**! 🚀