{"id":31667858,"url":"https://github.com/kuldeep-gif/interactive-gesture-speech-system","last_synced_at":"2026-04-09T12:04:06.781Z","repository":{"id":318524111,"uuid":"1071719158","full_name":"Kuldeep-gif/Interactive-Gesture-Speech-System","owner":"Kuldeep-gif","description":"An interactive AI system that translates real-time hand gestures into audible speech and converts spoken words into visual gestures using OpenCV and MediaPipe.","archived":false,"fork":false,"pushed_at":"2025-10-07T18:24:56.000Z","size":0,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-07T18:40:56.413Z","etag":null,"topics":["computer-vision","gesture-recognition","hci","machine-learning","mediapipe","opencv","python","scikit-learn","speech-recognition"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Kuldeep-gif.png","metadata":{"files":{"readme":"README(1).md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-07T18:08:10.000Z","updated_at":"2025-10-07T18:29:46.000Z","dependencies_parsed_at":"2025-10-07T18:41:00.312Z","dependency_job_id":null,"html_url":"https://github.com/Kuldeep-gif/Interactive-Gesture-Speech-System","commit_stats":null,"previous_names":["kuldeep-gif/interactive-gesture-speech-system"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Kuldeep-gif/Interactive-Gesture-Speech-System","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kuldeep-gif%2FInteractive-Gesture-Speech-System","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kuldeep-gif%2FInteractive-Gesture-Speech-System/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kuldeep-gif%2FInteractive-Gesture-Speech-System/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kuldeep-gif%2FInteractive-Gesture-Speech-System/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Kuldeep-gif","download_url":"https://codeload.github.com/Kuldeep-gif/Interactive-Gesture-Speech-System/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kuldeep-gif%2FInteractive-Gesture-Speech-System/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278872155,"owners_count":26060525,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","gesture-recognition","hci","machine-learning","mediapipe","opencv","python","scikit-learn","speech-recognition"],"created_at":"2025-10-08T00:48:34.621Z","updated_at":"2025-10-08T00:48:35.616Z","avatar_url":"https://github.com/Kuldeep-gif.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Interactive AI Gesture \u0026 Speech Recognition System\n\n\nA real-time, interactive AI system that translates hand gestures into audible speech and spoken words into visual gestures, creating a seamless, two-way communication experience.\n\n\n## 🌟 Features\n\nGesture-to-Speech: Performs a gesture and the system provides instant, synthesized speech feedback.\n\n* **Gesture-to-Speech**: Performs a gesture (e.g., thumbs up 👍) and the system responds with a synthesized voice (\"Great job!\").\n* **Speech-to-Gesture**: Speaks a command (e.g., \"Hello\") and the system displays a corresponding image of the gesture (e.g., a wave 👋).\n* **Real-Time Recognition**: Low-latency processing for a natural and seamless interaction.\n* **Multi-Hand Tracking**: Capable of recognizing gestures performed with one or two hands.\n* **Extendable**: Easily customizable to add new gestures and voice commands.\n\n## Tech Stack\n* **Programming Language**: Python\n* **Core Libraries**:\n    * **Computer Vision**: OpenCV, MediaPipe\n    * **Machine Learning**: scikit-learn\n    * **Speech Synthesis (TTS)**: gTTS, playsound\n    * **Speech Recognition (STT)**: SpeechRecognition, PyAudio\n## 🚀 Getting Started\nFollow these instructions to set up and run the project on your local machine.\n\n### 1. Prerequisites\n\n* Python 3.8+\n* A webcam and a microphone\n\n### 2. Installation \u0026 Setup\n\n**a. Project Structure:**\n\nProject Interactive-Gesture-Speech-System/\n│\n├── .venv/\n│\n├── data/\n│   ├── 0/\n│   │   ├── 0.jpg\n│   │   ├── 1.jpg\n│   │   └── ...\n│   ├── 1/\n│   │   └── ...\n│   └── (etc.)\n│\n├── gesture_images/\n│   ├── A.jpg\n│   ├── B.jpg\n│   ├── thumbs_up.jpg\n│   └── (etc.)\n│\n├── collect_imgs.py\n├── create_dataset.py\n├── train_classifier.py\n├── synapse_interactive.py\n│\n├── model.p\n├── data.pickle\n│\n└── README.md\n\n\n\n**b. Create and activate a virtual environment:**\n\n```bash\n# For Windows\npython -m venv .venv\n.venv\\Scripts\\activate\n```\n**c. Install the required libraries:**\n\n**Core Libraries**\n\nThese libraries provide the main functionalities for computer vision, machine learning, and user interaction.\n\n* **opencv-python:** The primary library for all computer vision tasks, including capturing webcam video and displaying images.\n\n* **mediapipe:** Used for real-time hand tracking and landmark detection.\n\n* **scikit-learn:** Used to train the RandomForestClassifier for gesture recognition.\n\n* **numpy:** A fundamental library for numerical operations, used to handle the data arrays.\n\n**Speech Functionality Libraries**\n\nThese libraries were added to handle the text-to-speech and speech-to-text features.\n\n* **gTTS:** (Google Text-to-Speech) Used to convert text phrases into audible speech.\n\n* **playsound:** A simple library used to play the audio files generated by gTTS.\n\n* **SpeechRecognition:** The main library for capturing microphone audio and converting speech to text.\n\n* **PyAudio:** Required by SpeechRecognition to access the microphone's audio stream.\n\n### 2. Folder Structure\n\nMake sure you create a folder named **gesture_images** and place one clear .jpg or .png file for each gesture you want to be displayed.\n## Usage\nThe project is divided into three main steps: data collection, model training, and running the interactive application:\n\n**1. Collect Gesture Data:** Run the **collect_imgs.py** script to capture images for your gestures. You will be prompted to **press 'q'** to start capturing for each class.\n\n```Bash\npython collect_imgs.py\n```\n**2. Create the Dataset:** Run the **create_dataset.py** script. This will process all the images in the data folder and create a data.pickle file.\n\n```Bash\npython create_dataset.py\n```\n**3. Train the Model:** Run the **train_classifier.py** script. This will use **data.pickle** to train the gesture recognition model and save it as **model.p**\n\n```Bash\npython train_classifier.py\n```\n**4.Run the Interactive Application:** You are now ready to run the main application!\n\n```Bash\npython synapse_interactive.py\n```\nLook at your webcam, perform gestures to hear the speech output, and speak commands to see the gesture images appear. Press 'q' to quit.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuldeep-gif%2Finteractive-gesture-speech-system","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkuldeep-gif%2Finteractive-gesture-speech-system","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuldeep-gif%2Finteractive-gesture-speech-system/lists"}