https://github.com/mmerlyn/asl-translator
Empowering the deaf and speech-impaired with a real-time ASL translator that converts hand gestures into English text using deep learning and computer vision.
https://github.com/mmerlyn/asl-translator
assistive-technology computer-vision deep-learning jupyter-notebook opencv python tensorflow
Last synced: about 2 months ago
JSON representation
Empowering the deaf and speech-impaired with a real-time ASL translator that converts hand gestures into English text using deep learning and computer vision.
- Host: GitHub
- URL: https://github.com/mmerlyn/asl-translator
- Owner: mmerlyn
- License: mit
- Created: 2024-11-10T02:18:36.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2026-02-08T01:30:11.000Z (5 months ago)
- Last Synced: 2026-02-12T21:37:29.908Z (4 months ago)
- Topics: assistive-technology, computer-vision, deep-learning, jupyter-notebook, opencv, python, tensorflow
- Language: Jupyter Notebook
- Homepage:
- Size: 16.4 MB
- Stars: 3
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# ASL-translator
## American Sign Language translator for the deaf and speech-impaired
## Project Objective
The goal of this project was to build a **sign language translator**. That is, an application capable of interpreting the hand gestures made by one user through a webcam, and translating that to text. We used the American Sign language for this purpose, to interpret English alphabets.
This system is primarily intended to support communication for individuals with speech or hearing impairments.
## Dataset:
- **Source:** [ASL Alphabet Dataset – Kaggle](https://www.kaggle.com/grassknoted/asl-alphabet)
- The dataset contains labeled images of American Sign Language alphabets for model training and testing.
## Technology used:
- **Python**
- **OpenCV** - for webcam capture and image preprocessing
- **TensorFlow & Keras** - for building and training the convolutional model
- **Jupyter Notebook** - for model development and evaluation
## How It Works
1. **Model Training:**
A CNN is trained on the ASL alphabet dataset to classify hand signs (A–Z).
2. **Video Feed & Preprocessing:**
Webcam frames are captured and preprocessed using OpenCV (background removal, thresholding) to isolate the hand gesture.
3. **Prediction:**
Each processed frame is passed into the CNN model to predict the corresponding alphabet.
4. **Display:**
The predicted alphabet is overlaid on the live webcam feed in real time.
The jupyter notebook contains the code for final model used and F1 scores for various classes. \n
The asl_abc video depict the model in action, on an youtube video (https://www.youtube.com/watch?v=pDfnf96qz_4)
## Results
- **Training Accuracy:** 99%
- **Test Accuracy:** ~60%
The convolution model achieved 99% training accuracy, while the test accuracy was lower, around 60%. On analysing the cause we found that this was mainly due to the quality difference between the well-thresholded training images and live webcam input, which contained more noise and variability.
## Learnings:
- Preprocessing consistency between training and live/test data is **crucial** for model performance.
- OpenCV is powerful for real-time computer vision but requires careful fine-tuning for dynamic environments.
- Deploying ML models in real-world settings highlights challenges that don’t appear during training.
## License
This project is licensed under the [MIT License](LICENSE).
## Acknowledgements
This project was completed as part of the Project-Based Learning (PBL) contest at **BMS Institute of Technology and Management**, Bengaluru, India.
**Team Members:**
1. Aishwarya M
2. Merlyn Mercylona Maki Reddy