{"id":20949293,"url":"https://github.com/amiriiw/speech_recognition","last_synced_at":"2025-03-13T05:15:41.426Z","repository":{"id":249757808,"uuid":"831621978","full_name":"amiriiw/speech_recognition","owner":"amiriiw","description":"Welcome to the Speech Recognition Project! This project is designed to train a model for recognizing voice commands and using it to control a simple Snake game in real-time. The project involves training a TensorFlow model for speech recognition and integrating it with a Pygame-based game.","archived":false,"fork":false,"pushed_at":"2024-08-12T05:59:25.000Z","size":33,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-19T22:25:57.227Z","etag":null,"topics":["key-word-recognition","numpy","pygame","queue","random","sounddevice","speech-recognition","tensorflow","threading"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/amiriiw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-21T05:55:31.000Z","updated_at":"2024-08-12T05:59:28.000Z","dependencies_parsed_at":"2024-08-12T07:04:37.181Z","dependency_job_id":null,"html_url":"https://github.com/amiriiw/speech_recognition","commit_stats":null,"previous_names":["amiriiw/speech_recognition"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amiriiw%2Fspeech_recognition","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amiriiw%2Fspeech_recognition/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amiriiw%2Fspeech_recognition/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amiriiw%2Fspeech_recognition/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/amiriiw","download_url":"https://codeload.github.com/amiriiw/speech_recognition/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243345615,"owners_count":20275872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["key-word-recognition","numpy","pygame","queue","random","sounddevice","speech-recognition","tensorflow","threading"],"created_at":"2024-11-19T00:35:01.658Z","updated_at":"2025-03-13T05:15:41.406Z","avatar_url":"https://github.com/amiriiw.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Speech Recognition Project\n\nWelcome to the **Speech Recognition Project**! This project is designed to train a model for recognizing voice commands and using it to control a simple Snake game in real-time. The project involves training a TensorFlow model for speech recognition and integrating it with a Pygame-based game.\n\n## Overview\n\nThis project consists of two main components:\n\n1. **speech_recognition_model_trainer.py**: This script is responsible for training a TensorFlow model to recognize voice commands. The model is trained using audio data to classify commands such as 'left,' 'right,' 'up,' and 'down.'\n2. **speech_recognition_game.py**: This script uses the trained model to control a Snake game based on voice commands. It captures audio from a microphone, processes it to make predictions, and translates those predictions into game controls.\n\n## Libraries Used\n\nThe following libraries are used in this project:\n\n- **[tensorflow](https://www.tensorflow.org/)**: TensorFlow is used for building and training the speech recognition model.\n- **[numpy](https://numpy.org/)**: NumPy is used for numerical operations and handling arrays.\n- **[pygame](https://www.pygame.org/docs/)**: Pygame is used to create the Snake game and handle graphics and user input.\n- **[sounddevice](https://python-sounddevice.readthedocs.io/en/0.4.7/)**: Sounddevice is used to capture audio from the microphone in real-time.\n- **[queue](https://docs.python.org/3/library/queue.html)**: The Queue module is used for handling audio data in a thread-safe manner.\n- **[random](https://docs.python.org/3/library/random.html)**: The Random module is used to generate random positions for the Snake game food.\n- **[threading](https://docs.python.org/3/library/threading.html)**: Threading is used to process audio data in parallel with the game loop.\n\n## Detailed Explanation\n\n### `speech_recognition_model_trainer.py`\n\nThis script is the core of the project, responsible for training the speech recognition model. The key components of the script are:\n\n- **TrainModel Class**: This class handles the training process for the speech recognition model. It includes methods to load datasets, prepare audio data, build the model, train the model, and evaluate its performance.\n  - **_load_commands() Method**: Loads command names from the dataset directory.\n  - **load_datasets() Method**: Loads and splits audio datasets into training, validation, and test sets.\n  - **_make_spec_ds() Method**: Converts audio data into spectrograms for model input.\n  - **build_model() Method**: Defines and compiles the CNN model architecture for speech recognition.\n  - **train_model() Method**: Trains the model on the prepared dataset.\n  - **evaluate_model() Method**: Evaluates the model's performance on the test set.\n- **ExportModel Class**: This class is used to export the trained model for inference. It includes methods to preprocess audio data and make predictions using the trained model.\n\n### `speech_recognition_game.py`\n\nThis script integrates the trained model with a real-time game controlled by voice commands. The key components of the script are:\n\n- **SnakeGame Class**: This class sets up and runs the Snake game using Pygame. It handles game initialization, drawing, scoring, and game logic.\n- **VoiceControl Class**: This class handles voice command processing and game control. It includes methods for audio processing, making predictions with the model, and translating predictions into game actions.\n  - **audio_callback() Method**: Captures audio data from the microphone and adds it to a queue.\n  - **process_audio() Method**: Processes audio data, makes predictions using the trained model, and updates the game state based on the predicted commands.\n\n### How It Works\n\n1. **Model Training**:\n    - The `speech_recognition_model_trainer.py` script reads audio data from the specified dataset directory.\n    - The audio is converted into spectrograms, and the model is trained to classify different voice commands.\n    - The trained model is saved for later use.\n\n2. **Voice-Controlled Game**:\n    - The `speech_recognition_game.py` script loads the trained model and starts a microphone stream to capture real-time audio.\n    - The captured audio is processed and passed through the model to predict voice commands.\n    - The predicted commands are used to control the Snake game in real-time.\n\n### Dataset\n\nThe dataset used for training the model can be accessed via this [Dataset](https://drive.google.com/drive/folders/1aAST8IX1-3Ri1eBdhq-4oyZVY8gjuuub?usp=sharing).\n\n## Installation and Setup\n\nTo use this project, follow these steps:\n\n1. Clone the repository:\n\n    ```bash\n    git clone https://github.com/amiriiw/speech_recognition\n    cd speech_recognition\n    ```\n\n2. Install the required libraries:\n\n    ```bash\n    pip install tensorflow numpy pygame sounddevice\n    ```\n\n3. Prepare your dataset (an audio dataset with voice commands).\n\n4. Train the model:\n\n    ```bash\n    python speech_recognition_model_trainer.py\n    ```\n\n5. Run the voice-controlled game:\n\n    ```bash\n    python speech_recognition_game.py\n    ```\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famiriiw%2Fspeech_recognition","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Famiriiw%2Fspeech_recognition","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famiriiw%2Fspeech_recognition/lists"}