https://github.com/s-m-j-i/multimodal-ser
An audio video-based multi-modal fusion approach for speech emotion recognition
https://github.com/s-m-j-i/multimodal-ser
Last synced: 2 months ago
JSON representation
An audio video-based multi-modal fusion approach for speech emotion recognition
- Host: GitHub
- URL: https://github.com/s-m-j-i/multimodal-ser
- Owner: S-M-J-I
- License: apache-2.0
- Created: 2023-04-22T06:36:49.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2024-11-21T07:34:02.000Z (6 months ago)
- Last Synced: 2024-11-21T08:26:36.024Z (6 months ago)
- Language: Jupyter Notebook
- Homepage:
- Size: 1.7 MB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# An audio video-based multi-modal fusion approach for speech emotion recognition
### Status: Submitted for Review
***Authors: S M JISHANUL ISLAM, SAHID HOSSAIN MUSTAKIM, MUSFIRAT HOSSAIN, MYSUN
MASHIRA, NUR ISLAM SHOURAV, MD. RAYHAN AHMED, SALEKUL ISLAM, A.K.M. MUZAHIDUL ISLAM, AND SWAKKHAR SHATABDA***
## Requirements
**Prerequisite:** A [CUDA-enabled GPU](https://gist.github.com/standaloneSA/99788f30466516dbcc00338b36ad5acf) is preferred. However, for those who would run this code on CPU, ensure to tweak the batch size in correspondence to your hardware capacity. Tweak the batch size in the [hyperparams.py](./src/utils/configs/hyperparams.py) file before running the notebooks. If the installations fail, kindly refer to: [conda instructions](./AlernateInstructions.md).
If frequent problems arise while running on the local environment, kindly resort to the [instructions for cloud notebooks](./cloud_notebooks/CloudInstructions.md), and run on any cloud platform.
**Step-2:** Clone this repository:
```sh
git clone https://github.com/S-M-J-I/Multimodal-Emotion-Recognition
```
If you have SSH configured:
```sh
git clone [email protected]:S-M-J-I/Multimodal-Emotion-Recognition.git
```
**Step-3:** Install pipenv. Skip if you already have it in your system.
```sh
pip3 install --user pipenv
```
**Step-4:** Install the modules. Run the following command in the terminal:
```sh
pipenv install -r requirements.txt
```
## Run the pipelines
To run the notebooks on SAVEE and RAVDESS, we recommend you download the dataset and unpack it in this directory. Then set the path to the directory in their respective notebooks.\
**Note: while setting the file path, ensure the exta '/' is added to the end. Example: `/path_to_dir/`**
To run the model on the datasets, navigate to the individual notebooks made for them in the [explore](./explore/) directory.
Run the following command in the terminal to start the local server:
```sh
pipenv run jupyter notebook
```
## Weights
To obtain the weights of the model, kindly access it through the [weights](./weights/) directory. ***Torch hub support for ease of model use is being worked on.***
#### For any assistance or issues, kindly open an Issue in this repository.
## Contributions
This repository is not accepting any contributors **OUTSIDE** the author list mentioned. For any issues related to the code, we request you to open an Issue.