{"id":14977268,"url":"https://github.com/prudhvignv/speech-emotion-recognization","last_synced_at":"2025-07-23T09:36:51.378Z","repository":{"id":50386834,"uuid":"287800619","full_name":"PrudhviGNV/Speech-Emotion-Recognization","owner":"PrudhviGNV","description":"Speech Emotion Detection using SVM, Decision Tree, Random Forest, MLP, CNN with different architectures","archived":false,"fork":false,"pushed_at":"2023-12-29T06:59:22.000Z","size":685,"stargazers_count":34,"open_issues_count":6,"forks_count":10,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-24T22:50:35.788Z","etag":null,"topics":["cnn","deep-learning","deep-neural-networks","ipynb","machine-learning-algorithms","mlp","ravdass","speech-emotion-detection","svm"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PrudhviGNV.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-15T18:13:08.000Z","updated_at":"2025-06-03T13:16:06.000Z","dependencies_parsed_at":"2024-09-18T21:07:13.639Z","dependency_job_id":"bc8de066-9229-4d64-a0e8-19574e72d91b","html_url":"https://github.com/PrudhviGNV/Speech-Emotion-Recognization","commit_stats":{"total_commits":19,"total_committers":1,"mean_commits":19.0,"dds":0.0,"last_synced_commit":"e5152745eff2a93c4840bcc7e16c0a164f72f076"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/PrudhviGNV/Speech-Emotion-Recognization","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PrudhviGNV%2FSpeech-Emotion-Recognization","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PrudhviGNV%2FSpeech-Emotion-Recognization/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PrudhviGNV%2FSpeech-Emotion-Recognization/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PrudhviGNV%2FSpeech-Emotion-Recognization/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PrudhviGNV","download_url":"https://codeload.github.com/PrudhviGNV/Speech-Emotion-Recognization/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PrudhviGNV%2FSpeech-Emotion-Recognization/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266655746,"owners_count":23963554,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cnn","deep-learning","deep-neural-networks","ipynb","machine-learning-algorithms","mlp","ravdass","speech-emotion-detection","svm"],"created_at":"2024-09-24T13:55:23.053Z","updated_at":"2025-07-23T09:36:51.355Z","avatar_url":"https://github.com/PrudhviGNV.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Speech Emotion Recognition using machine learning\n[![LICENCE.md](https://img.shields.io/github/license/PrudhviGNV/py-automl)](https://github.com/PrudhviGNV/py-automl/blob/master/LICENCE.md)[![Ask Me Anything !](https://img.shields.io/badge/Ask%20me-anything-1abc9c.svg)](https://GitHub.com/PrudhviGNV)[![Open Source Love svg2](https://badges.frapsoft.com/os/v2/open-source.svg?v=103)](https://github.com/PrudhviGNV/open-source-badges/)\n[![Awesome Badges](https://img.shields.io/badge/badges-awesome-green.svg)](https://github.com/PrudhviGNV/badges)\u003cbr\u003e\n--------------\n![python-mini-project-speech-emotion-recognition-1280x720](https://user-images.githubusercontent.com/39909903/91180489-9fe39400-e69c-11ea-9968-9adf6741d595.jpg)\n\n----------------\n## Overview:\n*  This project is completely based on  machine learning and deep learning where we train the models with RAVDESS Dataset which consists of audio files which are labelled with basic emotions.\n* #### This project is not just about to predict emotion based on the speech. and also to perform some analytical research by applying different machine learning algorithms and neural networks with different architectures.Finally  compare and analyse their results and to get beautiful insights.\n\n---------------------\n\n## Intro ..\nAs human beings speech is amongst the most natural way to express ourselves.As emotions play a vital role in communication, the detection and analysis of the same is of vital importance in today’s digital world of remote communication. Emotion detection is a challenging task, because emotions are subjective. There is no common consensus on how to measure or categorize them.\n\n----------\n\n### check out my [Medium blog](https://medium.com/@prudhvi.gnv/as-human-beings-speech-is-amongst-the-most-natural-way-to-express-ourselves-as-8fc38ebe1c44) for quick intuition and understanding\n\n------------\n\n## Dependencies:\n- #### python\n- #### librosa\n- #### soundfile\n- #### numpy\n- #### keras\n- #### sklearn\n- #### pandas\n\n---------------\n## Project Details\n   \nThe models which were discussed in the repository are MLP,SVM,Decision Tree,CNN,Random Forest and neural networks of mlp and CNN with different architectures.\n   - ### utilities.py                - Contains extraction of features,loading dataset functions\n   - ### loading_data.py         - Contains dataset loading,splitting data\n   - ### mlp_classifier_for_SER.py        - Contains mlp model code\n   - ### SER_using_ML_algorithms.py  - Contains SVM,randomforest,Decision tree Models.\n   - ### Speech_Emotion_Recognition_using_CNN.ipynb - Consists of CNN-1d model\n   \u003cbr\u003e\n   \n\u003cb\u003eNOTE :\u003c/b\u003e  Remaining .ipynb files were same as above files but shared from google colab.\n-----------------\n\n\n## Dataset Source - RAVDESS\n\u003cbr\u003e\n\nIn this project, I use  \u003ca href=\"https://zenodo.org/record/1188976#.Xl-poCEzZ0w\" \u003e RAVDESS\u003c/a\u003e dataset to train. \n\u003cbr\u003e\n\n![s1](https://user-images.githubusercontent.com/39909903/91179186-0798df80-e69b-11ea-824a-f2f65a7c082a.jpg)\n\n\n\u003cbr\u003e\nYou can find this dataset in kaggle or click on below link. \u003cbr\u003e\nhttps://www.kaggle.com/uwrfkaggler/ravdess-emotional-speech-audio\n\u003cbr\u003e\n\u003cbr\u003e\n\n\n2452 audio files, with 12 male speakers and 12 Female speakers, the lexical features (vocabulary) of the utterances are kept constant by speaking only 2 statements of equal lengths in 8 different emotions by all speakers.\nThis dataset was chosen because it consists of speech and song files classified by 247 untrained Americans to eight different emotions at two intensity levels: Calm, Happy, Sad, Angry, Fearful, Disgust, and Surprise, along with a baseline of Neutral for each actor.\n\n\u003ci\u003e\n   \u003cb\u003eprotip \u003cb/\u003e: if you are using google colabs. Use kaggle API to extract data from kaggle with super fast and with super ease :) \u003c/i\u003e\n\n\n\u003cH1\u003eData preprocessing : \u003c/H2\u003e\n\u003cp\u003e\n The heart of this project lies in preprocessing audio files. If you are able to do it . 70 % of project is already done.\n We take benefit of two packages which makes our task easier. \n -  \u003cB\u003eLibROSA \u003cB\u003e- for processing and extracting features from the audio file.\n -  \u003cb\u003e soundfile\u003cB\u003e - to read and write audio files in the storage.\n  \nThe main story in preprocessing audio files is to extract features from them.\n  \nFeatures supported:\n- MFCC (mfcc)\n- Chroma (chroma)\n- MEL Spectrogram Frequency (mel)\n- Contrast (contrast)\n- Tonnetz (tonnetz)\n\n\nIn this project, code related to preprocessing the dataset is written in two functions.\n- load_data()\n- extract_features()\n\n\n\n\nload_data() is used to traverse every file in a directory and we extract features from them and we prepare input and output data for mapping and feed to machine learning algorithms.\nand finally, we split the dataset into 80% training and 20% testing.\n\u003cbr\u003e\n\n```python\ndef load_data(test_size=0.2):\n  X, y = [], []\n  try :\n    for file in glob.glob(\"/content/drive/My Drive/wav/Actor_*/*.wav\"):\n          # get the base name of the audio file\n        basename = os.path.basename(file)\n        print(basename)\n          # get the emotion label\n        emotion = int2emotion[basename.split(\"-\")[2]]\n          # we allow only AVAILABLE_EMOTIONS we set\n        if emotion not in AVAILABLE_EMOTIONS:\n              continue\n          # extract speech features\n        features = extract_feature(file, mfcc=True, chroma=True, mel=True)\n          # add to data\n        X.append(features)\n        y.append(emotion)\n  except :\n       pass\n    # split the data to training and testing and return it\n  return train_test_split(np.array(X), y, test_size=test_size, random_state=7)\n  ```\n \n \n\n\u003cbr\u003eBelow is the code snippet to extract features from each file.\n\n  ```python\n  \ndef extract_feature(file_name, **kwargs):\n    \"\"\"\n    Extract feature from audio file `file_name`\n        Features supported:\n            - MFCC (mfcc)\n            - Chroma (chroma)\n            - MEL Spectrogram Frequency (mel)\n            - Contrast (contrast)\n            - Tonnetz (tonnetz)\n        e.g:\n        `features = extract_feature(path, mel=True, mfcc=True)`\n    \"\"\"\n    mfcc = kwargs.get(\"mfcc\")\n    chroma = kwargs.get(\"chroma\")\n    mel = kwargs.get(\"mel\")\n    contrast = kwargs.get(\"contrast\")\n    tonnetz = kwargs.get(\"tonnetz\")\n    with soundfile.SoundFile(file_name) as sound_file:\n        X = sound_file.read(dtype=\"float32\")\n        sample_rate = sound_file.samplerate\n        if chroma or contrast:\n            stft = np.abs(librosa.stft(X))\n        result = np.array([])\n        if mfcc:\n            mfccs = np.mean(librosa.feature.mfcc(y=X, sr=sample_rate, n_mfcc=40).T, axis=0)\n            result = np.hstack((result, mfccs))\n        if chroma:\n            chroma = np.mean(librosa.feature.chroma_stft(S=stft, sr=sample_rate).T,axis=0)\n            result = np.hstack((result, chroma))\n        if mel:\n            mel = np.mean(librosa.feature.melspectrogram(X, sr=sample_rate).T,axis=0)\n            result = np.hstack((result, mel))\n        if contrast:\n            contrast = np.mean(librosa.feature.spectral_contrast(S=stft, sr=sample_rate).T,axis=0)\n            result = np.hstack((result, contrast))\n        if tonnetz:\n            tonnetz = np.mean(librosa.feature.tonnetz(y=librosa.effects.harmonic(X), sr=sample_rate).T,axis=0)\n            result = np.hstack((result, tonnetz))\n    return result\n   \n   ```\n   Let's drive further into the project ..\n   \u003c/p\u003e\n   \n   --------\n   \n## Training and Analysis:\n\n\n\n### Traditional Machine Learning Models:\nPerforms different traditional algorithms such as -Decision Tree, SVM, Random forest .\n\n Refer \u003cbr\u003e\n - [SER_using_ML_algorithms.py](https://github.com/PrudhviGNV/SpeechEmotionRecognization/blob/master/SER%20using%20ML%20algorithms.py) or \n - [Speech Emotion Recognition using ML - SVM, DT, Random Forest.ipynb](https://github.com/PrudhviGNV/SpeechEmotionRecognization/blob/master/Speech%20Emotion%20Recognition%20using%20ML%20-%20SVM%2C%20DT%2C%20Random%20Forest.ipynb)\n \n Finds that these algorithms don’t give satisfactory results. So Deep Learning comes into action.\n### Deep Learning:\nimplements classical neural network architecture such as mlp \nRefer \u003cbr\u003e\n- [mlp classifier for SER.py](https://github.com/PrudhviGNV/SpeechEmotionRecognization/blob/master/mlp%20classifier%20for%20SER.py) or \n- [Speech Emotion Recognition using MLP.ipynb](https://github.com/PrudhviGNV/SpeechEmotionRecognization/blob/master/Speech%20Emotion%20Recognition%20using%20MLP.ipynb) \n\nFound that Deep learning algorithms like mlp tends to overfit to the data. So the preferred neural network is CNN which is a game changer in many fields and applications.\nWants to perform some analysis to find the best CNN architecture for available dataset.\nHere CNN with different architectures is trained against the dataset and the accuracy is recorded.\nHere every architecture has same configuration and is trained to 500 epochs.\n\nRefer \u003cbr\u003e\n- [Speech Emotion Recognition using CNN.ipynb](https://github.com/PrudhviGNV/SpeechEmotionRecognization/blob/master/Speech%20Emotion%20Recognition%20using%20CNN.ipynb)\n\n### Visualization.\nfor better understanding about data and also for  visualizing waveform and spectogram of audio files.\nRefer \u003cbr\u003e\n- [emotion_spectogram_CNN_2D.ipynb](https://github.com/PrudhviGNV/SpeechEmotionRecognization/blob/master/emotion_spectogram_CNN_2D.ipynb)\n\n\n----------------\n---------------\n\n## Conclusion and Analysis : \n - #### Neural networks performs better than traditional classical machine learning models in maximun cases ( by compare metrics) \n - #### Since Deep learning models are data hunger .. They tend overfit the training data.  (if we keep on training the model. we get 95% +  accuracy :) )\n - #### CNN architectures performs better than traditional neural network architectures. (cnn in most cases perform better than mlp under same configuration)\n - #### CNN with different architectures with same configuration , with same learning rate, with same number of epochs  also have vast difference in the accuracy (from [Speech_Emotion_Recognition_using_CNN.ipynb](https://github.com/PrudhviGNV/SpeechEmotionRecognization/blob/master/Speech%20Emotion%20Recognition%20using%20CNN.ipynb) )\n\n   \n \n \n -------------\n --------------\n \n## Hope you like this project :)\n\n-----------\n## LICENSE:\n[MIT](https://github.com/PrudhviGNV/SpeechEmotionRecognization/blob/master/LICENSE)\n\n --------\n## Contact:\n\u003ca href=\"https://www.linkedin.com/in/prudhvignv\"\u003e\u003cimg src=\"https://github.com/PrudhviGNV/PrudhviGNV/blob/master/logos/linkedin.png\" width=\"40\" /\u003e \u003c/a\u003e  \u003ca href=\"https://github.com/PrudhviGNV\"\u003e\u003cimg src=\"https://github.com/PrudhviGNV/PrudhviGNV/blob/master/logos/github-logo.png\" width=\"40\" /\u003e \u003c/a\u003e  \u003ca href=\"https://www.facebook.com/prudhvi.gnv/\"\u003e\u003cimg src=\"https://github.com/PrudhviGNV/PrudhviGNV/blob/master/logos/facebook.png\" width=\"40\" /\u003e \u003c/a\u003e  \u003ca href=\"mailto:prudhvi.gnv@gmail.com\"\u003e\u003cimg src=\"https://github.com/PrudhviGNV/PrudhviGNV/blob/master/logos/google-plus.png\" width=\"40\" /\u003e \u003c/a\u003e  \u003ca\n href=\"https://www.instagram.com/prudhvi_gnv\"\u003e\u003cimg src=\"https://github.com/PrudhviGNV/PrudhviGNV/blob/master/logos/instagram.png\" width=\"40\" /\u003e \u003c/a\u003e\u003ca href=\"https://prudhvignv.github.io\"\u003e\u003cimg src=\"https://github.com/PrudhviGNV/PrudhviGNV/blob/master/logos/home.png\" width=\"40\" /\u003e\u003c/a\u003e\n \n \n\n\n\n \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprudhvignv%2Fspeech-emotion-recognization","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprudhvignv%2Fspeech-emotion-recognization","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprudhvignv%2Fspeech-emotion-recognization/lists"}