https://github.com/lucidrains/aoa-pytorch
A Pytorch implementation of Attention on Attention module (both self and guided variants), for Visual Question Answering
https://github.com/lucidrains/aoa-pytorch
attention attention-mechanism captioning visual-question-answering vqa
Last synced: 5 months ago
JSON representation
A Pytorch implementation of Attention on Attention module (both self and guided variants), for Visual Question Answering
- Host: GitHub
- URL: https://github.com/lucidrains/aoa-pytorch
- Owner: lucidrains
- License: mit
- Created: 2020-11-07T19:04:22.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2020-11-08T00:17:30.000Z (almost 5 years ago)
- Last Synced: 2025-03-08T05:12:19.234Z (7 months ago)
- Topics: attention, attention-mechanism, captioning, visual-question-answering, vqa
- Language: Python
- Homepage:
- Size: 39.1 KB
- Stars: 41
- Watchers: 2
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## Attention on Attention - Pytorch
A Pytorch implementation of the Attention on Attention module, from the paper An Improved Attention for Visual Question Answering. The repository will include both the Self and Guided (cross-attention) variants.
## Install
```bash
$ pip install aoa-pytorch
```
## Usage
Self Attention on Attention
```python
import torch
from aoa_pytorch import AoA
attn = AoA(
dim = 512,
heads = 8
)
x = torch.randn(1, 1024, 512)
attn(x) + x # (1, 1024, 512)
```
Guided Attention on Attention
```python
```python
import torch
from aoa_pytorch import AoA
attn = AoA(
dim = 512,
heads = 8
)
x = torch.randn(1, 1024, 512)
context = torch.randn(1, 1024, 512)
attn(x, context = context) + x # (1, 1024, 512)
```
## Citations
```bibtex
@misc{rahman2020improved,
title = {An Improved Attention for Visual Question Answering},
author = {Tanzila Rahman and Shih-Han Chou and Leonid Sigal and Giuseppe Carenini},
year = {2020},
eprint = {2011.02164},
archivePrefix = {arXiv},
primaryClass = {cs.CV}
}
```
```bibtex
@misc{huang2019attention,
title = {Attention on Attention for Image Captioning},
author = {Lun Huang and Wenmin Wang and Jie Chen and Xiao-Yong Wei},
year = {2019},
eprint = {1908.06954},
archivePrefix = {arXiv},
primaryClass = {cs.CV}
}
```