https://github.com/imvision12/nextvit-tf

A Tensorflow implementation of "Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios"
https://github.com/imvision12/nextvit-tf

keras tensorflow transformer vision-transformer

Last synced: 29 days ago
JSON representation

A Tensorflow implementation of "Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios"

• Host: GitHub
• URL: https://github.com/imvision12/nextvit-tf
• Owner: IMvision12
• Created: 2023-03-06T21:09:36.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2023-04-02T08:54:12.000Z (almost 2 years ago)
• Last Synced: 2024-11-14T11:37:47.188Z (3 months ago)
• Topics: keras, tensorflow, transformer, vision-transformer
• Language: Python
• Homepage:
• Size: 410 KB
• Stars: 4
• Watchers: 3
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# NextViT-tf

This repository is about an implementation of the research paper "Next-ViT: Next Generation Vision Transformer for Efficient Deployment in

Realistic Industrial Scenarios" using `Tensorflow`.

In this paper authors propose a next generation vision Transformer for efficient deployment in realistic industrial scenarios, namely Next-ViT, which dominates both CNNs and ViTs from the perspective of latency/accuracy trade-off. In this work, the Next Convolution Block (NCB) and Next Transformer Block (NTB) are respectively developed to capture local and global information with deployment-friendly mechanisms. Then, Next Hybrid Strategy (NHS) is designed to stack NCB and NTB in an efficient hybrid paradigm, which boosts performance in various downstream tasks. Extensive experiments show that Next-ViT significantly outperforms existing CNNs, ViTs and CNN-Transformer hybrid architectures with respect to the latency/accuracy trade-off across various vision tasks.



  



# Model Architecture



  



# Usage:

Clone Github Repo: 

```bash

$ git clone https://github.com/IMvision12/NextViT-tf

$ cd nextvit_tf

```

Then import model

```py

import tensorflow as tf

from models.nextvit_tf import nextvit_base

model = nextvit_base(input_shape=(224, 224, 3), num_classes=100)

print(model.summary(expand_nested=True))

```

# References

[1] NextViT paper: https://arxiv.org/pdf/2207.05501v4

[2] Official NextViT Repo: https://github.com/bytedance/next-vit