https://github.com/doubledaibo/compcaption_neurips2018

A Neural Compositional Paradigm for Image Captioning
https://github.com/doubledaibo/compcaption_neurips2018

compositionality image-captioning neurips-2018 nlp

Last synced: 14 days ago
JSON representation

A Neural Compositional Paradigm for Image Captioning

• Host: GitHub
• URL: https://github.com/doubledaibo/compcaption_neurips2018
• Owner: doubledaibo
• License: other
• Created: 2018-10-18T04:20:09.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2019-04-03T11:28:32.000Z (over 5 years ago)
• Last Synced: 2024-07-31T07:16:10.382Z (3 months ago)
• Topics: compositionality, image-captioning, neurips-2018, nlp
• Language: Lua
• Homepage:
• Size: 903 KB
• Stars: 9
• Watchers: 2
• Forks: 0
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-diverse-captioning - [lua-code

README

        
# Code of [A Neural Compositional Paradigm for Image Captioning](http://papers.nips.cc/paper/7346-a-neural-compositional-paradigm-for-image-captioning)

Borrowed some code from [AdaptiveAttention](https://github.com/jiasenlu/AdaptiveAttention).

## Overview

![Overview](compcap.png)

## Summarization

- We propose a compositional captioning model, where caption generation follows a recursive structure, 

which naturally fits the properties of human language. 

Specifically,

we could generate diverse captions based on the same visual semantics by changing syntax,

or attentions. 

![Good](samples.png)

## Codes

- *nlp*: scripts for processing sentences using a nlp toolkit

- *data*: scripts for processing data

- *misc*: models, dataloaders, and auxiliary codes

- *image_models*: pretrained image models

- *test*: scripts for studies

- *scripts*: bashs

## Citation

```

@inproceedings{dai2018aneural,

title = {A Neural Compositional Paradigm for Image Captioning},

author = {Dai, Bo and Fidler, Sanja and Lin, Dahua},

booktitle = {Advances in Neural Information Processing Systems 31},

pages = {658--668},

year = {2018}

}

```