{"id":31642275,"url":"https://github.com/manishms18/sign-language-advance","last_synced_at":"2025-10-07T03:57:58.191Z","repository":{"id":313987836,"uuid":"1053711063","full_name":"Manishms18/Sign-Language-Advance","owner":"Manishms18","description":"Contributed to a vision-driven accessibility tool translating sign language into text","archived":false,"fork":false,"pushed_at":"2025-09-26T03:57:22.000Z","size":56,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-26T05:45:41.277Z","etag":null,"topics":["accessibility","ai-for-accessibility","american-sign-language","asl","cohens-kappa","confusion-matrix","data-augmentation","deep-learning","efficientnet","f1-score","gru","keras","multimodal-fusion","recurrent-neural-networks","resnet50","robustness","sign-language-recognition","tensorflow","transfer-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Manishms18.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-09T20:22:43.000Z","updated_at":"2025-09-26T03:57:25.000Z","dependencies_parsed_at":"2025-09-09T23:54:27.253Z","dependency_job_id":"ea297a79-c343-4952-a46c-415203356481","html_url":"https://github.com/Manishms18/Sign-Language-Advance","commit_stats":null,"previous_names":["manishms18/sign-language-advance"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Manishms18/Sign-Language-Advance","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Manishms18%2FSign-Language-Advance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Manishms18%2FSign-Language-Advance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Manishms18%2FSign-Language-Advance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Manishms18%2FSign-Language-Advance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Manishms18","download_url":"https://codeload.github.com/Manishms18/Sign-Language-Advance/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Manishms18%2FSign-Language-Advance/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278717449,"owners_count":26033542,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accessibility","ai-for-accessibility","american-sign-language","asl","cohens-kappa","confusion-matrix","data-augmentation","deep-learning","efficientnet","f1-score","gru","keras","multimodal-fusion","recurrent-neural-networks","resnet50","robustness","sign-language-recognition","tensorflow","transfer-learning"],"created_at":"2025-10-07T03:57:48.854Z","updated_at":"2025-10-07T03:57:58.187Z","avatar_url":"https://github.com/Manishms18.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Advancing Sign Language Interpretation with Transfer Learning and Multimodal Features\n\n- Baseline CNN for sign language gesture recognition\n- Transfer learning using ResNet-50 and EfficientNet-B0\n- Hand landmark extraction via MediaPipe (GRU-based)\n- Comprehensive evaluation: accuracy, macro/micro F1, Cohen's kappa, confusion matrix\n- Robustness testing (lighting, noise)\n\nThe repository includes:\n\n- Scripts for training and evaluation.\n- Sample outputs (results.json, plots).\n\n## Quickstart\n\n```bash\npython -m venv .venv \u0026\u0026 source .venv/bin/activate\npip install -r requirements.txt\n\n# Train baseline CNN on mini-sample\npython -m src.models.train_cnn --config configs/config.yaml\npython -m src.eval.evaluate --config configs/config.yaml --weights outputs/baseline_cnn_best.h5\npython -m src.eval.robustness --config configs/config.yaml --weights outputs/baseline_cnn_best.h5\n\n# Transfer Learning\npython -m src.models.train_transfer --config configs/config.yaml --backbone resnet50\npython -m src.models.train_transfer --config configs/config.yaml --backbone efficientnetb0\n\n# Landmark + GRU (stub example)\npython -m src.models.train_landmark_gru --config configs/config.yaml\n\n## Demo\n\n### Streamlit (browser)\nTo run a simple browser-based demo using Streamlit:\n```bash\npip install -r requirements.txt\nstreamlit run demo/streamlit_app.py\n```\nUpload a hand image (PNG or JPEG) and optionally adjust the weights path. The app displays the predicted letter and confidence as well as per-class probabilities.\n\n### Gradio (browser)\nTo run a Gradio demo:\n```bash\npip install -r requirements.txt\npython demo/gradio_app.py\n```\nThis serves a web interface where you can upload an image and see the probability distribution across classes and the top prediction.\n\n### Webcam (local)\nTo run a live webcam demo (requires a webcam):\n```bash\npython scripts/webcam_demo.py\n```\n\n\n### Citation \nIf you are using this work, please cite using below details :\n```\n@misc{shukla_gupta_2025_advancing_sign_language,\n  author       = {Shukla, Manish and Gupta, Harsh},\n  title        = {Advancing Sign Language Interpretation with Transfer Learning and Multimodal Features},\n  year         = {2025},\n  month        = sep,\n  note         = {Preprint (Version 1), Research Square},\n  doi          = {10.21203/rs.3.rs-7586144/v1},\n  url          = {https://doi.org/10.21203/rs.3.rs-7586144/v1}\n}\n```\nShukla, M., \u0026 Gupta, H. (2025, September 12). Advancing Sign Language Interpretation with Transfer Learning and Multimodal Features (Version 1) [Preprint]. Research Square. https://doi.org/10.21203/rs.3.rs-7586144/v1\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanishms18%2Fsign-language-advance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmanishms18%2Fsign-language-advance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanishms18%2Fsign-language-advance/lists"}