Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/showlab/assistgaze
https://github.com/showlab/assistgaze
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/showlab/assistgaze
- Owner: showlab
- Created: 2023-10-19T08:25:19.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-06-25T14:02:53.000Z (6 months ago)
- Last Synced: 2024-10-17T01:05:27.912Z (3 months ago)
- Language: Python
- Size: 1.28 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# [EMNLP23 Poster]GazeVQA
This is the official repository which provides a baseline model for our proposed task: GazeVQA: A Video Question Answering Dataset for Multiview Eye-Gaze Task-Oriented Collaborations.
[[Paper]](https://aclanthology.org/2023.emnlp-main.648/)
Model Architecture (see [[Paper]](https://arxiv.org/abs/2203.04203) for details):
## Install
(1) PyTorch. See https://pytorch.org/ for instruction. For example,
```
conda install pytorch torchvision torchtext cudatoolkit=11.3 -c pytorch
```
(2) PyTorch Lightning. See https://www.pytorchlightning.ai/ for instruction. For example,
```
python -m pip install lightning
```
## Data
The released dataset is under this repository
[[Dataset]](https://github.com/mfurkanilaslan/GazeVQA)
The processed data can be downloaded from the link
[[processed_data]](https://drive.google.com/drive/folders/1fSp3Umi8mfxljGXOoOKssthgRHEmpx8H?usp=sharing)
## Encoding
Before starting, you should encode the instructional videos, scripts, QAs.
## Training & Evaluation
Just run the code with single GPU. The code will automatically process training and evalutaion process.
```
python train.py
```
## Contact
Feel free to contact us if you have any problems: [email protected], or leave an issue in this repo.