https://github.com/ay-ritesh/sign-language-recognition
Crafted a deep learning-based system for real-time sign language recognition, leveraging computer vision and neural networks. The system translates hand gestures into text to bridge communication gaps for the hearing-impaired.
https://github.com/ay-ritesh/sign-language-recognition
cnn keras nlp opencv python real-time-detection tenserflow yolov5
Last synced: about 1 month ago
JSON representation
Crafted a deep learning-based system for real-time sign language recognition, leveraging computer vision and neural networks. The system translates hand gestures into text to bridge communication gaps for the hearing-impaired.
- Host: GitHub
- URL: https://github.com/ay-ritesh/sign-language-recognition
- Owner: Ay-ritesh
- Created: 2025-02-23T09:29:14.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2025-02-23T09:37:22.000Z (over 1 year ago)
- Last Synced: 2025-03-04T07:15:43.412Z (over 1 year ago)
- Topics: cnn, keras, nlp, opencv, python, real-time-detection, tenserflow, yolov5
- Language: Python
- Homepage:
- Size: 93.5 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# 🤟 An Enhanced Deep Learning Approach for American Sign Language (ASL) Recognition
A real-time sign language recognition system using **YOLOv5 + CNN + OpenCV**, designed to translate hand gestures into readable text.
The project helps bridge the communication gap between **sign language users and verbal speakers**, promoting inclusivity in healthcare, education, and public services.
---
## 👨🔬 Team Details
| Name | Reg. No. | Contribution |
|------|----------|--------------|
| Aayush Amritesh | 21BCE2331 | Data Preprocessing, Model Training & Testing, Evaluation |
| Dev Chandrakar | 21BCE2960 | Literature Survey, Dataset Collection, Documentation |
| Aditya Kumar Jha | 21BCE3759 | Model Selection, UI/Backend Integration |
🧑🏫 **Guided by:** *Dr. Viswanathan A., Associate Professor (Sr.)*
School of Computer Science and Engineering, VIT
---
## 🎯 Objective
Develop a **real-time ASL Recognition System** that:
- Detects gestures using **YOLOv5**
- Classifies gestures using **CNN**
- Translates gestures to text through a simple user interface
- Supports low‐cost consumer hardware like webcams & laptops
---
## 🔍 Problem Motivation
Over **70M people** rely on sign language. Most verbal speakers don’t understand it, causing a *communication barrier* in critical areas such as hospitals, classrooms, and public services.
Deep learning enables **accessible, affordable, and real-time solutions** for gesture recognition.
---
## 🏗️ System Architecture
**Modules:**
1. 📷 *Data Acquisition* – Webcam or image input
2. 🧹 *Preprocessing (OpenCV)* – Background removal, segmentation
3. 🎯 *Gesture Detection (YOLOv5)* – Identify hand gesture
4. 🧠 *Classification (CNN)* – Recognize gesture category
5. 💬 *Sign-to-Text Translation* – Output real-time prediction
6. 🖥️ *User Interface* – Minimal & user-friendly
---
## 🧰 Tools & Technologies
| Category | Tools |
|----------|-------|
| Programming | Python |
| Deep Learning | YOLOv5, CNN (TensorFlow/PyTorch) |
| Image Processing | OpenCV |
| UI/Development | Jupyter/Colab/VS Code |
| Hardware | Standard webcam, i5/8GB RAM laptop |
---
## 📊 Dataset & Parameters
| Feature | Value |
|--------|--------|
| Image Size | 224×224 |
| Classes | 6 (Hello, Yes, No, Thanks, Please, I Love You) |
| Model | YOLOv5s + Custom CNN |
| Optimizer | Adam |
| Latency | **< 100ms real-time** |
---
## 🚀 Result Summary
✔ **Real-time gesture detection**
✔ Works effectively in **low-light & diverse backgrounds**
✔ Runs smoothly on **consumer-grade hardware**
✔ Achieved **95% accuracy (static)** & **90% dynamic** in adequate lighting conditions
---
## 🏁 Conclusion
The system successfully bridges the communication gap by converting American Sign Language (ASL) gestures to readable text in real time using a hybrid deep learning model (YOLOv5 + CNN). It is affordable, portable, and scalable, making it suitable for deployment in schools, hospitals, and public service centers. Future work includes sentence-level recognition, multi-language support, and mobile deployment.
---
## 🔮 Future Enhancements
- 📌 Support for complete sentence-level gesture recognition
- 🌎 Extension to multiple sign languages (e.g., ISL, BSL, etc.)
- 📱 Mobile application deployment on Android/iOS
- 🎙️ Voice output for recognized text (sign-to-speech)
- 🧠 Use of Transformer-based or sequence models for better temporal gesture understanding
---