https://github.com/vision-cair/chatcaptioner

Official Repository of ChatCaptioner
https://github.com/vision-cair/chatcaptioner

Last synced: 1 day ago
JSON representation

Official Repository of ChatCaptioner

• Host: GitHub
• URL: https://github.com/vision-cair/chatcaptioner
• Owner: Vision-CAIR
• License: mit
• Created: 2023-03-10T18:36:12.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-04-13T20:05:09.000Z (over 1 year ago)
• Last Synced: 2024-10-18T21:17:36.053Z (21 days ago)
• Language: Jupyter Notebook
• Size: 36.2 MB
• Stars: 451
• Watchers: 4
• Forks: 27
• Open Issues: 5
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.md

Awesome Lists containing this project

• awesome-ChatGPT-repositories - ChatCaptioner - Official Repository of ChatCaptioner (Others)

README

        
# Interactive ChatCaptioner for image and video

Official repository of **ChatCaptioner** and **Video ChatCaptioner**.

ChatCaptioner paper [ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions](https://arxiv.org/abs/2303.06594) 

Video ChatCaptioner paper [Video ChatCaptioner: Towards the Enriched Spatiotemporal Descriptions](https://arxiv.org/abs/2304.04227)

## Demo

![demo1](ChatCaptioner/demo_pic/demo1.gif)

![demo2](ChatCaptioner/demo_pic/demo2.gif)

![demo3](Video_ChatCaptioner/demo_pic/dance.gif)

![demo4](Video_ChatCaptioner/demo_pic/skating.gif)

* ChatCaptiners:

     * ChatCaptioner for images: [ChatCaptioner](ChatCaptioner/README.md)

     * ChatCaptioner for videos: [Video ChatCaptioner](Video_ChatCaptioner/README.md)

## Acknowledgement

+ [ChatGPT](https://openai.com/blog/chatgpt/)

+ [BLIP2](https://huggingface.co/docs/transformers/main/model_doc/blip-2)

Please cite ChatCaptioner and Video ChatCaptioner from the following bibtex

```

@article{zhu2023chatgpt,

  title={ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions},

  author={Deyao Zhu and Jun Chen and Kilichbek Haydarov and Xiaoqian Shen and Wenxuan Zhang and Mohamed Elhoseiny},

  journal={arXiv preprint arXiv:2303.06594},

  year={2023}

}

```

```

@article{chen2023video,

      title={Video ChatCaptioner: Towards the Enriched Spatiotemporal Descriptions}, 

      author={Jun Chen and Deyao Zhu and Kilichbek Haydarov and Xiang Li and Mohamed Elhoseiny},

      journal={arXiv preprint arXiv:2304.04227},

      year={2023}

}

```

## *License*

ChatCaptioner and Video ChatCaptioner are released under the [MIT license](LICENSE).