https://github.com/ihp-lab/mm_analysis_empathy
[ICMI 23] Multimodal Analysis and Assessment of Therapist Empathy in Motivational Interviews
https://github.com/ihp-lab/mm_analysis_empathy
motivational-interviewing multimodal-learning therapist-empathy
Last synced: about 1 year ago
JSON representation
[ICMI 23] Multimodal Analysis and Assessment of Therapist Empathy in Motivational Interviews
- Host: GitHub
- URL: https://github.com/ihp-lab/mm_analysis_empathy
- Owner: ihp-lab
- Created: 2023-08-04T21:40:29.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2024-02-29T01:02:14.000Z (over 2 years ago)
- Last Synced: 2025-03-27T12:11:18.649Z (about 1 year ago)
- Topics: motivational-interviewing, multimodal-learning, therapist-empathy
- Language: Python
- Homepage:
- Size: 229 KB
- Stars: 6
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Multimodal Analysis and Assessment of Therapist Empathy in Motivational Interviews
Trang Tran,
Yufeng Yin,
Leili Tavabi,
Joannalyn Delacruz,
Brian Borsari,
Joshua Woolley,
Stefan Scherer,
Mohammad Soleymani
USC ICT, San Francisco VAHCS, UCSF Psychiatry and Behavioral Sciences
ICMI 2023
## Introduction
This is the official implementation of our ICMI 2023 paper: Multimodal Analysis and Assessment of Therapist Empathy in Motivational Interviews.
TODO
## Installation
Clone repo:
```
git clone https://github.com/ihp-lab/mm_analysis_empathy.git
cd mm_analysis_empathy
```
The code is tested with Python == 3.10, PyTorch == 1.11.0 and CUDA == 11.3 on NVIDIA Quadro RTX 8000. We recommend you to use [anaconda](https://www.anaconda.com/) to manage dependencies.
```
conda create -n mm_empathy python=3.10
conda activate mm_empathy
conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3 -c pytorch
pip install pandas
pip install -U scikit-learn
pip install transformers==4.28.1
```
## Data
Sample data can be downloaded from Google Drive: [sample data](https://drive.google.com/file/d/1PeMqm-2xohMnnlUr4M-THWi94765lot1/view?usp=drive_link). Since the clinical data is not public, we have sampled a subset in addition to replacing a random set of transcripts with random tokens. The cross-validation folds are also for sample/toy use only.
Download and untar the file, then put `sample_data` under `./mm_analysis_empathy`.
## Checkpoints
Checkpoints are available on Google Drive: [exps_independent](https://drive.google.com/drive/folders/1f0CH87HKSoCYKLfvH28nhuSSQF9O-OZE?usp=drive_link) and [exps_dependent](https://drive.google.com/drive/folders/1oqUx5y1_0V59zQ9Qsw31fD-a0dCB284-?usp=sharing).
Put `./exps_independent` and `./exps_dependent` under `./mm_analysis_empathy`.
## Training and Evaluation
### General notes:
* The json file `./sample_data/sample_data_folds.json` should be your cross-validation folds; supply the appropriate file depending on the therapist-independent vs. therapist-dependent settings.
* Quartiles ara 0-indexed, i.e. to train/evaluate on Q2 (like in our paper), set argument `--quartile 1`. `quartile=-1` uses the full sessions.
* The code assumes acoustic features are stored under `./sample_data/sample_features`
```
CUDA_VISIBLE_DEVICES=0 python run.py --model {text,audio,early_fusion_finetune,late_fusion_finetune}
--model_name {text,audio,early_fusion_finetune,late_fusion_finetune}
--by_speaker {therapist,both}
--quartile {0,1,2,3,-1}
--output_filename {therapist,both}-quartile-{0,1,2,3,all}
--epochs_num 5
--out_path ./exps_{dependent,independent}
--data_root ./sample_data
--data_path ./sample_data/sample_data_feats.tsv
--dataset_fold_path ./sample_data/sample_data_folds.json
```
For example, this is the command to train and evaluate on the unimodal text model, using only the therapist turns from Q2, assuming `sample_data_folds.json` contains cross-validation folds for the therapist-dependent setting:
```
CUDA_VISIBLE_DEVICES=0 python run.py --model text --model_name text
--by_speaker therapist
--quartile 1
--output_filename therapist-quartile-1
--epochs_num 5
--out_path ./exps_dependent
--data_root ./sample_data
--data_path ./sample_data/sample_data_feats.tsv
--dataset_fold_path ./sample_data/sample_data_folds.json
```
Get overall f1 scores across folds
```
python compute_overall_scores.py --results PATH_TO_CSV_FILE
```
csv files are saved under `./OUT_PATH/MODEL_NAME/OUTPUT_FILENAME`.
## Citation
TODO
## Contact
If you have any questions, please raise an issue or email to Trang Tran (`ttran@ict.usc.edu`).
## Credits
Our codes are based on the following repositories.
- [text-empathy-recognition](https://github.com/ihp-lab/empathy-recognition-acii-2023)
- [multimodal-fusion](https://github.com/ihp-lab/XNorm)