https://github.com/lucidrains/cross-transformers-pytorch

Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch
https://github.com/lucidrains/cross-transformers-pytorch

artificial-intelligence attention-mechanism deep-learning few-shot-learning transformers

Last synced: 12 days ago
JSON representation

Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch

• Host: GitHub
• URL: https://github.com/lucidrains/cross-transformers-pytorch
• Owner: lucidrains
• License: mit
• Created: 2020-12-11T17:17:19.000Z (almost 4 years ago)
• Default Branch: main
• Last Pushed: 2021-03-30T04:04:32.000Z (over 3 years ago)
• Last Synced: 2024-11-02T02:42:08.335Z (19 days ago)
• Topics: artificial-intelligence, attention-mechanism, deep-learning, few-shot-learning, transformers
• Language: Python
• Homepage:
• Size: 90.8 KB
• Stars: 51
• Watchers: 5
• Forks: 11
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        


## Cross Transformers - Pytorch (wip)

Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch

## Install

```bash

$ pip install cross-transformers-pytorch

```

## Usage

```python

import torch

from torch import nn

import torch.nn.functional as F

from torchvision import models

from cross_transformers_pytorch import CrossTransformer

resnet = models.resnet34(pretrained = True)

model = nn.Sequential(*[*resnet.children()][:-2])

cross_transformer = CrossTransformer(

    dim = 512,

    dim_key = 128,

    dim_value = 128

)

# (batch, channels, height, width)

img_query = torch.randn(1, 3, 224, 224)

# (batch, classes, num supports, channels, height, width)

img_supports = torch.randn(1, 2, 4, 3, 224, 224)

labels = torch.randint(0, 2, (1,))

dists = cross_transformer(model, img_query, img_supports) # (1, 2)

loss = F.cross_entropy(dists, labels)

loss.backward()

```

## Citations

```bibtex

@misc{doersch2020crosstransformers,

    title={CrossTransformers: spatially-aware few-shot transfer}, 

    author={Carl Doersch and Ankush Gupta and Andrew Zisserman},

    year={2020},

    eprint={2007.11498},

    archivePrefix={arXiv},

    primaryClass={cs.CV}

}

```