https://github.com/anarya22/sign-language-recognition
Real-time hybrid sign language recognition system using Random Forest and LSTM for gesture-to-text conversion with dynamic sentence formation.
https://github.com/anarya22/sign-language-recognition
lstm-model mediapipe opencv sign-language-recognition-system
Last synced: 3 months ago
JSON representation
Real-time hybrid sign language recognition system using Random Forest and LSTM for gesture-to-text conversion with dynamic sentence formation.
- Host: GitHub
- URL: https://github.com/anarya22/sign-language-recognition
- Owner: Anarya22
- Created: 2026-04-12T04:48:14.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2026-04-12T05:07:10.000Z (3 months ago)
- Last Synced: 2026-04-12T07:06:55.331Z (3 months ago)
- Topics: lstm-model, mediapipe, opencv, sign-language-recognition-system
- Language: Jupyter Notebook
- Homepage: https://drive.google.com/file/d/1ymYsBiK-jUF7YlUZU3SUgYJ1Fv2GpNql/view?usp=sharing
- Size: 2.64 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# 🤟 Real-Time Sign Language Recognition System
> A hybrid AI system that converts hand gestures into text in real time — combining static and motion-based recognition for dynamic sentence formation.
---
## 📌 Project Overview
This project presents a **real-time Sign Language Recognition system** that converts hand gestures into text. It follows a **hybrid AI approach** combining:
- **Static gesture recognition** — for alphabets, numbers, and space
- **Motion-based gesture recognition** — for commonly used words
The system uses **hand landmark detection** to extract features and performs real-time inference using a webcam, making it interactive and practical for communication assistance.
---
## ✨ Key Features
- 🔄 Real-time gesture-to-text conversion
- 🧠 Hybrid model (Static + Motion recognition)
- 📝 Sentence formation from predicted gestures
- 🔀 Mode switching (Static / Motion)
- ⌨️ Interactive controls (clear, backspace, delete word)
---
## 🛠️ Tech Stack
| Tool | Purpose |
|------|---------|
| Python | Core language |
| OpenCV | Video capture & processing |
| MediaPipe | Hand landmark detection |
| Scikit-learn (Random Forest) | Static gesture classification |
| TensorFlow / Keras (LSTM) | Motion gesture classification |
| NumPy | Feature array processing |
---
## ⚙️ How It Works
### 🟢 Static Gesture Recognition
- Extracts **21 hand landmarks** using MediaPipe
- Converts them into **42 normalized features** (x, y coordinates)
- Uses a **Random Forest classifier** for predicting alphabets, numbers, and space
### 🔵 Motion Gesture Recognition
- Captures sequences of **30 frames**
- Each frame contains **42 features**
- Uses an **LSTM model** to learn temporal patterns
- Predicts dynamic gestures like words (e.g., `HELLO`, `YES`, `OK`, `NO`, `THANKS`)
### 🟣 Hybrid System
- Combines both models into a **single inference pipeline**
- Manual mode switching ensures stable predictions
- Outputs are combined to form **meaningful sentences**
---
## 📊 Evaluation
The models were evaluated using:
- Accuracy
- Precision
- Recall
- F1-score
The system achieved **high accuracy in controlled environments**. Results highlight the importance of diverse datasets for better real-world generalization.
---
## 👤 My Contributions
- Designed and implemented **landmark-based feature extraction pipeline** (42 features per frame)
- Trained **Random Forest model** for static gesture recognition
- Developed **LSTM-based sequence model** for motion gesture recognition
- Engineered **hybrid inference pipeline** combining static and temporal models
- Implemented evaluation metrics (accuracy, precision, recall, F1-score)
- Contributed to **dataset creation and preprocessing**
---
## 👥 Team Credits
This project was developed collaboratively by a **team of four members** as part of a final-year undergraduate project at **Heritage Institute of Technology, Kolkata**.
I primarily worked on **model training and dataset creation**, while other team members contributed to data collection, system integration, and user interface development.
---
## ⚠️ Limitations & Future Work
**Current Limitations:**
- Uses ASL due to limited ISL datasets
- Performance may vary across lighting conditions and users
**Future Work:**
- [ ] Extend to Indian Sign Language (ISL)
- [ ] Improve dataset diversity
- [ ] Deploy as a web application
---
## 🎥 Demo
▶️ [Watch Demo Video](https://drive.google.com/file/d/1ymYsBiK-jUF7YlUZU3SUgYJ1Fv2GpNql/view?usp=sharing)
---
## 🚀 How to Run
```bash
# Clone the repository
git clone https://github.com/Anarya22/Sign-Language-Recognition
# Navigate to project folder
cd Sign-Language-Recognition
# Install dependencies
pip install -r requirements.txt
# Run inference
python inference_classifier.py
```
---
## ⭐ Final Note
This project demonstrates the integration of **computer vision** and **machine learning** to build an interactive, real-time AI system. It highlights the importance of combining **spatial and temporal modeling** for solving practical problems.
---
Made with ❤️ at Heritage Institute of Technology, Kolkata