{"id":48828506,"url":"https://github.com/anarya22/sign-language-recognition","last_synced_at":"2026-04-14T19:02:40.808Z","repository":{"id":350796118,"uuid":"1208297791","full_name":"Anarya22/Sign-Language-Recognition","owner":"Anarya22","description":"Real-time hybrid sign language recognition system using Random Forest and LSTM for gesture-to-text conversion with dynamic sentence formation.","archived":false,"fork":false,"pushed_at":"2026-04-12T05:07:10.000Z","size":2772,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-12T07:06:55.331Z","etag":null,"topics":["lstm-model","mediapipe","opencv","sign-language-recognition-system"],"latest_commit_sha":null,"homepage":"https://drive.google.com/file/d/1ymYsBiK-jUF7YlUZU3SUgYJ1Fv2GpNql/view?usp=sharing","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Anarya22.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-12T04:48:14.000Z","updated_at":"2026-04-12T05:10:24.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Anarya22/Sign-Language-Recognition","commit_stats":null,"previous_names":["anarya22/sign-language-recognition"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Anarya22/Sign-Language-Recognition","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anarya22%2FSign-Language-Recognition","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anarya22%2FSign-Language-Recognition/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anarya22%2FSign-Language-Recognition/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anarya22%2FSign-Language-Recognition/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Anarya22","download_url":"https://codeload.github.com/Anarya22/Sign-Language-Recognition/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anarya22%2FSign-Language-Recognition/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31810741,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-14T18:05:02.291Z","status":"ssl_error","status_checked_at":"2026-04-14T18:05:01.765Z","response_time":153,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lstm-model","mediapipe","opencv","sign-language-recognition-system"],"created_at":"2026-04-14T19:02:23.871Z","updated_at":"2026-04-14T19:02:40.804Z","avatar_url":"https://github.com/Anarya22.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🤟 Real-Time Sign Language Recognition System\n\n\u003e A hybrid AI system that converts hand gestures into text in real time — combining static and motion-based recognition for dynamic sentence formation.\n\n---\n\n## 📌 Project Overview\n\nThis project presents a **real-time Sign Language Recognition system** that converts hand gestures into text. It follows a **hybrid AI approach** combining:\n\n- **Static gesture recognition** — for alphabets, numbers, and space\n- **Motion-based gesture recognition** — for commonly used words\n\nThe system uses **hand landmark detection** to extract features and performs real-time inference using a webcam, making it interactive and practical for communication assistance.\n\n---\n\n## ✨ Key Features\n\n- 🔄 Real-time gesture-to-text conversion\n- 🧠 Hybrid model (Static + Motion recognition)\n- 📝 Sentence formation from predicted gestures\n- 🔀 Mode switching (Static / Motion)\n- ⌨️ Interactive controls (clear, backspace, delete word)\n\n---\n\n## 🛠️ Tech Stack\n\n| Tool | Purpose |\n|------|---------|\n| Python | Core language |\n| OpenCV | Video capture \u0026 processing |\n| MediaPipe | Hand landmark detection |\n| Scikit-learn (Random Forest) | Static gesture classification |\n| TensorFlow / Keras (LSTM) | Motion gesture classification |\n| NumPy | Feature array processing |\n\n---\n\n## ⚙️ How It Works\n\n### 🟢 Static Gesture Recognition\n- Extracts **21 hand landmarks** using MediaPipe\n- Converts them into **42 normalized features** (x, y coordinates)\n- Uses a **Random Forest classifier** for predicting alphabets, numbers, and space\n\n### 🔵 Motion Gesture Recognition\n- Captures sequences of **30 frames**\n- Each frame contains **42 features**\n- Uses an **LSTM model** to learn temporal patterns\n- Predicts dynamic gestures like words (e.g., `HELLO`, `YES`, `OK`, `NO`, `THANKS`)\n\n### 🟣 Hybrid System\n- Combines both models into a **single inference pipeline**\n- Manual mode switching ensures stable predictions\n- Outputs are combined to form **meaningful sentences**\n\n---\n\n## 📊 Evaluation\n\nThe models were evaluated using:\n\n- Accuracy\n- Precision\n- Recall\n- F1-score\n\nThe system achieved **high accuracy in controlled environments**. Results highlight the importance of diverse datasets for better real-world generalization.\n\n---\n\n## 👤 My Contributions\n\n- Designed and implemented **landmark-based feature extraction pipeline** (42 features per frame)\n- Trained **Random Forest model** for static gesture recognition\n- Developed **LSTM-based sequence model** for motion gesture recognition\n- Engineered **hybrid inference pipeline** combining static and temporal models\n- Implemented evaluation metrics (accuracy, precision, recall, F1-score)\n- Contributed to **dataset creation and preprocessing**\n\n---\n\n## 👥 Team Credits\n\nThis project was developed collaboratively by a **team of four members** as part of a final-year undergraduate project at **Heritage Institute of Technology, Kolkata**.\n\nI primarily worked on **model training and dataset creation**, while other team members contributed to data collection, system integration, and user interface development.\n\n---\n\n## ⚠️ Limitations \u0026 Future Work\n\n**Current Limitations:**\n- Uses ASL due to limited ISL datasets\n- Performance may vary across lighting conditions and users\n\n**Future Work:**\n- [ ] Extend to Indian Sign Language (ISL)\n- [ ] Improve dataset diversity\n- [ ] Deploy as a web application\n\n---\n\n## 🎥 Demo\n\n▶️ [Watch Demo Video](https://drive.google.com/file/d/1ymYsBiK-jUF7YlUZU3SUgYJ1Fv2GpNql/view?usp=sharing)\n\n---\n\n## 🚀 How to Run\n\n```bash\n# Clone the repository\ngit clone https://github.com/Anarya22/Sign-Language-Recognition\n\n# Navigate to project folder\ncd Sign-Language-Recognition\n\n# Install dependencies\npip install -r requirements.txt\n\n# Run inference\npython inference_classifier.py\n```\n\n---\n\n## ⭐ Final Note\n\nThis project demonstrates the integration of **computer vision** and **machine learning** to build an interactive, real-time AI system. It highlights the importance of combining **spatial and temporal modeling** for solving practical problems.\n\n---\n\n\u003cp align=\"center\"\u003eMade with ❤️ at Heritage Institute of Technology, Kolkata\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanarya22%2Fsign-language-recognition","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanarya22%2Fsign-language-recognition","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanarya22%2Fsign-language-recognition/lists"}