https://github.com/nvlabs/segformer

Official PyTorch implementation of SegFormer
https://github.com/nvlabs/segformer

ade20k cityscapes semantic-segmentation transformer

Last synced: 3 days ago
JSON representation

Official PyTorch implementation of SegFormer

• Host: GitHub
• URL: https://github.com/nvlabs/segformer
• Owner: NVlabs
• License: other
• Created: 2021-06-11T17:22:07.000Z (over 3 years ago)
• Default Branch: master
• Last Pushed: 2024-08-02T15:50:33.000Z (5 months ago)
• Last Synced: 2024-11-27T22:00:46.239Z (24 days ago)
• Topics: ade20k, cityscapes, semantic-segmentation, transformer
• Language: Python
• Homepage: https://arxiv.org/abs/2105.15203
• Size: 2.57 MB
• Stars: 2,594
• Watchers: 31
• Forks: 357
• Open Issues: 107
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
[![NVIDIA Source Code License](https://img.shields.io/badge/license-NSCL-blue.svg)](https://github.com/NVlabs/SegFormer/blob/master/LICENSE)

![Python 3.8](https://img.shields.io/badge/python-3.8-green.svg)

# SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers



  





  Figure 1: Performance of SegFormer-B0 to SegFormer-B5.



### [Project page](https://github.com/NVlabs/SegFormer) | [Paper](https://arxiv.org/abs/2105.15203) | [Demo (Youtube)](https://www.youtube.com/watch?v=J0MoRQzZe8U) | [Demo (Bilibili)](https://www.bilibili.com/video/BV1MV41147Ko/) | [Intro Video](https://www.youtube.com/watch?v=nBjXyoltCHU)

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers.


[Enze Xie](https://xieenze.github.io/), [Wenhai Wang](https://whai362.github.io/), [Zhiding Yu](https://chrisding.github.io/), [Anima Anandkumar](http://tensorlab.cms.caltech.edu/users/anima/), [Jose M. Alvarez](https://rsu.data61.csiro.au/people/jalvarez/), and [Ping Luo](http://luoping.me/).


NeurIPS 2021.

This repository contains the official Pytorch implementation of training & evaluation code and the pretrained models for [SegFormer](https://arxiv.org/abs/2105.15203).

SegFormer is a simple, efficient and powerful semantic segmentation method, as shown in Figure 1.

We use [MMSegmentation v0.13.0](https://github.com/open-mmlab/mmsegmentation/tree/v0.13.0) as the codebase.

🔥🔥 SegFormer is on [MMSegmentation](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/segformer). 🔥🔥 

## Installation

For install and data preparation, please refer to the guidelines in [MMSegmentation v0.13.0](https://github.com/open-mmlab/mmsegmentation/tree/v0.13.0).

Other requirements:

```pip install timm==0.3.2```

An example (works for me): ```CUDA 10.1``` and  ```pytorch 1.7.1``` 

```

pip install torchvision==0.8.2

pip install timm==0.3.2

pip install mmcv-full==1.2.7

pip install opencv-python==4.5.1.48

cd SegFormer && pip install -e . --user

```

## Evaluation

Download `trained weights`. 

(

[google drive](https://drive.google.com/drive/folders/1GAku0G0iR9DsBxCbfENWMJ27c5lYUeQA?usp=sharing) | 

[onedrive](https://connecthkuhk-my.sharepoint.com/:f:/g/personal/xieenze_connect_hku_hk/Ept_oetyUGFCsZTKiL_90kUBy5jmPV65O5rJInsnRCDWJQ?e=CvGohw)

)

Example: evaluate ```SegFormer-B1``` on ```ADE20K```:

```

# Single-gpu testing

python tools/test.py local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py /path/to/checkpoint_file

# Multi-gpu testing

./tools/dist_test.sh local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py /path/to/checkpoint_file 

# Multi-gpu, multi-scale testing

tools/dist_test.sh local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py /path/to/checkpoint_file  --aug-test

```

## Training

Download `weights` 

(

[google drive](https://drive.google.com/drive/folders/1b7bwrInTW4VLEm27YawHOAMSMikga2Ia?usp=sharing) | 

[onedrive](https://connecthkuhk-my.sharepoint.com/:f:/g/personal/xieenze_connect_hku_hk/EvOn3l1WyM5JpnMQFSEO5b8B7vrHw9kDaJGII-3N9KNhrg?e=cpydzZ)

) 

pretrained on ImageNet-1K, and put them in a folder ```pretrained/```.

Example: train ```SegFormer-B1``` on ```ADE20K```:

```

# Single-gpu training

python tools/train.py local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py 

# Multi-gpu training

./tools/dist_train.sh local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py 

```

## Visualize

Here is a demo script to test a single image. More details refer to [MMSegmentation's Doc](https://mmsegmentation.readthedocs.io/en/latest/get_started.html).

```shell

python demo/image_demo.py ${IMAGE_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${DEVICE_NAME}] [--palette-thr ${PALETTE}]

```

Example: visualize ```SegFormer-B1``` on ```CityScapes```: 

```shell

python demo/image_demo.py demo/demo.png local_configs/segformer/B1/segformer.b1.512x512.ade.160k.py \

/path/to/checkpoint_file --device cuda:0 --palette cityscapes

```

## License

Please check the LICENSE file. SegFormer may be used non-commercially, meaning for research or 

evaluation purposes only. For business inquiries, please visit our website and submit the form: [NVIDIA Research Licensing](https://www.nvidia.com/en-us/research/inquiries/).

## Citation

```

@inproceedings{xie2021segformer,

  title={SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers},

  author={Xie, Enze and Wang, Wenhai and Yu, Zhiding and Anandkumar, Anima and Alvarez, Jose M and Luo, Ping},

  booktitle={Neural Information Processing Systems (NeurIPS)},

  year={2021}

}

```