https://github.com/declare-lab/misa
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis
https://github.com/declare-lab/misa
Last synced: 8 months ago
JSON representation
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis
- Host: GitHub
- URL: https://github.com/declare-lab/misa
- Owner: declare-lab
- License: mit
- Created: 2020-06-16T04:25:34.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2023-03-14T15:13:34.000Z (over 3 years ago)
- Last Synced: 2025-04-10T03:52:06.191Z (about 1 year ago)
- Language: Python
- Size: 229 KB
- Stars: 229
- Watchers: 4
- Forks: 33
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis
Code for the [ACM MM 2020](https://2020.acmmm.org) paper [MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis](https://arxiv.org/pdf/2005.03545.pdf)
### Setup the environment
We work with a conda environment.
```
conda env create -f environment.yml
conda activate misa-code
```
### Data Download
- Install [CMU Multimodal SDK](https://github.com/A2Zadeh/CMU-MultimodalSDK). Ensure, you can perform ```from mmsdk import mmdatasdk```.
- Option 1: Download [pre-computed splits](https://drive.google.com/drive/folders/1IBwWNH0XjPnZWaAlP1U2tIJH6Rb3noMI?usp=sharing) and place the contents inside ```datasets``` folder.
- Option 2: Re-create splits by downloading data from MMSDK. For this, simply run the code as detailed next.
### Running the code
1. ```cd src```
2. Set ```word_emb_path``` in ```config.py``` to [glove file](http://nlp.stanford.edu/data/glove.840B.300d.zip).
3. Set ```sdk_dir``` to the path of CMU-MultimodalSDK.
2. ```python train.py --data mosi```. Replace ```mosi``` with ```mosei``` or ```ur_funny``` for other datasets.
### Citation
If this paper is useful for your research, please cite us at:
```
@article{hazarika2020misa,
title={MISA: Modality-Invariant and-Specific Representations for Multimodal Sentiment Analysis},
author={Hazarika, Devamanyu and Zimmermann, Roger and Poria, Soujanya},
journal={arXiv preprint arXiv:2005.03545},
year={2020}
}
```
### Contact
For any questions, please email at [hazarika@comp.nus.edu.sg](mailto:hazarika@comp.nus.edu.sg)