https://github.com/pleaseconnectwifi/DANCE

PyTorch code for Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles (DANCE)
https://github.com/pleaseconnectwifi/DANCE

Last synced: about 2 months ago
JSON representation

PyTorch code for Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles (DANCE)

• Host: GitHub
• URL: https://github.com/pleaseconnectwifi/DANCE
• Owner: pleaseconnectwifi
• Created: 2022-11-29T12:08:46.000Z (about 2 years ago)
• Default Branch: master
• Last Pushed: 2022-11-29T19:00:49.000Z (about 2 years ago)
• Last Synced: 2024-08-01T02:25:45.679Z (4 months ago)
• Size: 2.24 MB
• Stars: 24
• Watchers: 3
• Forks: 0
• Open Issues: 2
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• Awesome-Reasoning-Foundation-Models - [Code

README

        
# Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles

![](imgs/fig1.gif)

### [Paper (ArXiv)]() | [Project Page](https://shuquanye.com/DANCE_website) | [Pre-trained Models]()

[Shuquan Ye](https://shuquanye.com/)2,[Yujia Xie](https://sites.google.com/view/yujia?pli=1)1,[Dongdong Chen](https://www.dongdongchen.bid/)1,  [Yichong Xu](https://xycking.wixsite.com/yichongxu)1,   [Lu Yuan](https://www.microsoft.com/en-us/research/people/luyuan/)1,  [Chenguang Zhu](https://www.microsoft.com/en-us/research/people/chezhu/)1,  [Jing Liao](https://liaojing.github.io/html/)2  

1Microsoft, 2City University of Hong Kong




This is the PyTorch code of the [DANCE](https://shuquanye.com/DANCE_website) [\[paper\]](). The code is on PyTorch 1.11. Pre-training with ours code requires 4 nodes each with 8 A100 GPUs.

Catalog:

- [ ] Code for DANCE-augmented Pre-training

- [ ] Code for DANCE-augmented Fine-tuning

- [ ] Code for Image-Text Retrieval, OK-VQA

- [ ] Download of Pre-trained and Fine-tuned Checkpoints 

## BibTeX

```

```