{"id":32650361,"url":"https://github.com/andercruz/audio-classification-neural-networks-cnn","last_synced_at":"2026-04-29T23:05:18.999Z","repository":{"id":321312152,"uuid":"1085363389","full_name":"AnderCruz/Audio-Classification-Neural-Networks-CNN","owner":"AnderCruz","description":"This project explores various approaches for audio classification using neural networks with TensorFlow and Keras. The notebook demonstrates the complete process from data loading and preprocessing to model building, training, evaluation, and inference.","archived":false,"fork":false,"pushed_at":"2025-10-29T00:39:24.000Z","size":5888,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"Main","last_synced_at":"2025-10-29T02:25:04.715Z","etag":null,"topics":["audio-classification","audio-processing","cnn","deep-learning","environmental-sound-classification","keras","machine-learning","neural-networks","spectrogram-analysis","speech-recognition","tensorflow","transfer-learning","yamnet"],"latest_commit_sha":null,"homepage":"https://audio-classification-model.streamlit.app/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AnderCruz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-29T00:02:17.000Z","updated_at":"2025-10-29T00:39:28.000Z","dependencies_parsed_at":"2025-10-29T02:25:29.658Z","dependency_job_id":"703b9c50-58b1-465a-af44-a3b43d5cd6e7","html_url":"https://github.com/AnderCruz/Audio-Classification-Neural-Networks-CNN","commit_stats":null,"previous_names":["andercruz/audio-classification-neural-networks"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/AnderCruz/Audio-Classification-Neural-Networks-CNN","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnderCruz%2FAudio-Classification-Neural-Networks-CNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnderCruz%2FAudio-Classification-Neural-Networks-CNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnderCruz%2FAudio-Classification-Neural-Networks-CNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnderCruz%2FAudio-Classification-Neural-Networks-CNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AnderCruz","download_url":"https://codeload.github.com/AnderCruz/Audio-Classification-Neural-Networks-CNN/tar.gz/refs/heads/Main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnderCruz%2FAudio-Classification-Neural-Networks-CNN/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32447312,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-29T22:27:22.272Z","status":"ssl_error","status_checked_at":"2026-04-29T22:10:49.234Z","response_time":110,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio-classification","audio-processing","cnn","deep-learning","environmental-sound-classification","keras","machine-learning","neural-networks","spectrogram-analysis","speech-recognition","tensorflow","transfer-learning","yamnet"],"created_at":"2025-10-31T07:06:31.161Z","updated_at":"2026-04-29T23:05:18.994Z","avatar_url":"https://github.com/AnderCruz.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Audio Classification with Neural Networks\n\nThis project provides a comprehensive exploration of various neural network approaches for audio classification using TensorFlow and Keras. The implementation covers the complete machine learning pipeline from data loading and preprocessing to model deployment and inference.\n\n## Project Overview\n\nThe notebook demonstrates multiple state-of-the-art techniques for audio classification, comparing different architectural approaches and their performance on speech command recognition tasks.\n\n## Key Features \u0026 Techniques\n\n### Data Handling \u0026 Preprocessing\n- **Audio Loading**: Reading WAV files from compressed archives (.gz, .tar formats)\n- **Signal Processing**: Resampling audio to consistent 16kHz sample rate using SciPy\n- **Normalization**: Padding/trimming audio to fixed length (16,000 samples)\n- **Efficient Pipelines**: Using `tf.data.Dataset` for optimized data loading with shuffling, batching, and prefetching\n- **Label Encoding**: Converting string labels to integers using scikit-learn's LabelEncoder\n\n### Model Architectures\n\n#### 1. Time-Domain Classification (1D CNN)\n- **Input**: Raw audio waveforms (16,000 samples × 1 channel)\n- **Architecture**:\n  - Conv1D (16 filters, kernel_size=3, ReLU activation)\n  - MaxPooling1D (pool_size=2)\n  - Flatten layer\n  - Dense (64 units, ReLU)\n  - Output (36 units, Softmax)\n- **Parameters**: ~4.1 million trainable parameters\n- **Use Case**: Direct learning from raw audio signals\n\n#### 2. Frequency-Domain Classification (2D CNN \u0026 Spectrograms)\n- **Input**: Spectrograms generated via Short-Time Fourier Transform (STFT)\n- **Processing**: Converting time-domain signals to frequency-domain representations\n- **Architecture**: 2D convolutional layers adapted for spectrogram input\n- **Enhancements**: Custom normalization layers and attention mechanisms\n\n#### 3. Attention Mechanisms\n- **Custom Implementation**: ChannelAttention layer for Keras\n- **Integration**: Enhanced 2D CNN architecture with attention gates\n- **Benefits**: Improved feature focus and model interpretability\n\n#### 4. Transfer Learning with YAMNet\n- **Base Model**: Pre-trained YAMNet audio event classification model from TensorFlow Hub\n- **Feature Extraction**: Using YAMNet embeddings (1,024 dimensions)\n- **Custom Head**: Training new classification layers on top of frozen embeddings\n- **Dataset Adaptation**: Applied to ESC-50 environmental sound classification dataset\n\n## Dataset Information\n\n### Speech Commands Dataset\n- **36 Audio Classes**: \n  - Basic commands: \"yes\", \"no\", \"stop\", \"go\", \"up\", \"down\", \"left\", \"right\"\n  - Numbers: \"zero\" through \"nine\"\n  - Animals: \"bird\", \"dog\", \"cat\"\n  - Household: \"bed\", \"house\", \"tree\"\n  - Miscellaneous: \"happy\", \"wow\", \"follow\", \"learn\", \"visual\", etc.\n  - Background noise category\n\n### ESC-50 Dataset (for Transfer Learning)\n- Environmental Sound Classification dataset\n- 50 classes of environmental recordings\n- Used for YAMNet transfer learning experiments\n\n## Technical Implementation\n\n### Preprocessing Pipeline\n```python\ndef load_and_process_audio(filename, max_length=16000):\n    # Read and decode WAV file\n    # Resample to 16kHz using SciPy\n    # Pad/trim to fixed length\n    # Return normalized tensor\n```\n\n### Training Configuration\n- **Optimizer**: Adam\n- **Loss Function**: Sparse Categorical Crossentropy\n- **Metrics**: Accuracy\n- **Batch Size**: 32\n- **Validation Split**: 20% stratified split\n- **Epochs**: 10+ with early stopping potential\n\n### Model Evaluation\n- Training/validation accuracy and loss tracking\n- Visualization of learning curves\n- Confusion matrix analysis\n- Performance comparison across architectures\n\n## Usage Examples\n\n### Basic Training\n```python\n# Time-domain model training\nhistory_time_domain = model_time_domain.fit(\n    train_dataset,\n    epochs=10,\n    batch_size=32,\n    validation_data=val_dataset\n)\n```\n\n### Spectrogram Generation\n```python\n# Convert audio to spectrograms\nspectrogram = tf.signal.stft(audio, frame_length=255, frame_step=128)\n```\n\n### Transfer Learning\n```python\n# Load pre-trained YAMNet\nyamnet_model = hub.load('https://tfhub.dev/google/yamnet/1')\n# Extract embeddings and train custom classifier\n```\n\n## Performance Metrics\n\nThe project includes comprehensive evaluation of:\n- **Training Accuracy**: Model performance on training data\n- **Validation Accuracy**: Generalization capability\n- **Loss Curves**: Training stability and convergence\n- **Inference Speed**: Real-time classification potential\n- **Model Size**: Parameter efficiency\n\n## 🎮 Inference \u0026 Deployment\n\n### Single Audio Prediction\n```python\ndef predict_audio_class(model, audio_path):\n    # Preprocess audio\n    # Run model inference\n    # Return class probabilities and predicted label\n```\n\n### Model Saving\n```python\nmodel_spectrogram.save('audio_classification_model.h5')\n```\n\n## Requirements\n\n### Core Dependencies\n```\ntensorflow\u003e=2.8.0\nnumpy\u003e=1.21.0\nscipy\u003e=1.7.0\nscikit-learn\u003e=1.0.0\nmatplotlib\u003e=3.5.0\nlibrosa\u003e=0.9.0\n```\n\n### Optional Dependencies\n```\ntensorflow-hub  # For YAMNet transfer learning\nipython         # For notebook visualization\n```\n\n## Project Structure\n\n```\naudio_classification/\n├── Audio_Classification.ipynb      # Main notebook\n├── data_audio/\n│   └── dataset_commands.gz         # Compressed dataset\n├── models/\n│   ├── time_domain_model.h5        # Saved 1D CNN model\n│   └── spectrogram_model.h5        # Saved 2D CNN model\n└── utils/\n    └── audio_processing.py         # Helper functions\n```\n\n## Tags\n\n`audio-classification` `neural-networks` `tensorflow` `keras` `cnn` `spectrogram-analysis` `transfer-learning` `yamnet` `speech-recognition` `machine-learning` `deep-learning` `audio-processing` `python` `1d-cnn` `2d-cnn` `attention-mechanism` `signal-processing` `audio-ml` `environmental-sound-classification` `speech-commands` `esc-50` `data-augmentation` `tf-data-pipeline`\n\n## Research Applications\n\nThis project demonstrates practical implementations of:\n- Multi-modal neural network architectures for audio\n- Comparative analysis of time-domain vs frequency-domain approaches\n- Effective transfer learning strategies for audio tasks\n- Attention mechanisms for improved feature learning\n- Production-ready data preprocessing pipelines\n\n## Potential Extensions\n\n- Real-time audio classification\n- Mobile deployment with TensorFlow Lite\n- Multi-label audio classification\n- Audio generation and style transfer\n- Cross-modal learning (audio + text)\n\n## References\n\n- TensorFlow Audio Recognition Tutorials\n- YAMNet: Pre-trained audio event classifier\n- Speech Commands Dataset (Google)\n- ESC-50 Dataset for environmental sound classification\n- Attention mechanisms in audio processing literature\n\n*This project serves as both an educational resource and a practical foundation for building production audio classification systems.*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandercruz%2Faudio-classification-neural-networks-cnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fandercruz%2Faudio-classification-neural-networks-cnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandercruz%2Faudio-classification-neural-networks-cnn/lists"}