https://github.com/YoadTew/zero-shot-video-to-text

https://github.com/YoadTew/zero-shot-video-to-text

Last synced: 9 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/YoadTew/zero-shot-video-to-text
• Owner: YoadTew
• Created: 2022-07-15T10:47:02.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2022-10-22T08:34:09.000Z (about 2 years ago)
• Last Synced: 2024-08-01T13:29:22.628Z (3 months ago)
• Language: Python
• Size: 1.51 MB
• Stars: 73
• Watchers: 2
• Forks: 7
• Open Issues: 6
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Official Implementation of [Zero-Shot Video Captioning with Evolving Pseudo-Tokens](https://arxiv.org/abs/2207.11100)

## Approach

![](git_images/Architecture.png)

## Example of capabilities

![](git_images/Examples.png)

## Usage

### To run captioning on a video:

```bash

$ python run.py 

--token_wise--randomized_prompt

--run_type caption_videos

--data_path examples/example_video.mp4

```

### To run captioning on a single image:

```bash

$ python run.py 

--token_wise --randomized_prompt

--run_type caption_images

--data_path examples/example_image.jpg

```

## Citation

Please cite our work if you use it in your research:

```

@article{tewel2022videocap,

  title={Zero-Shot Video Captioning with Evolving Pseudo-Tokens},

  author={Tewel, Yoad and Shalev, Yoav and Nadler, Roy and Schwartz, Idan and Wolf, Lior},

  journal={arXiv preprint arXiv:2207.11100},

  year={2022}

}

```