https://github.com/yeliudev/catnet
🛰️ Learning to Aggregate Multi-Scale Context for Instance Segmentation in Remote Sensing Images (TNNLS 2024)
https://github.com/yeliudev/catnet
instance-segmentation object-detection
Last synced: about 1 month ago
JSON representation
🛰️ Learning to Aggregate Multi-Scale Context for Instance Segmentation in Remote Sensing Images (TNNLS 2024)
- Host: GitHub
- URL: https://github.com/yeliudev/catnet
- Owner: yeliudev
- License: gpl-3.0
- Created: 2021-11-23T13:30:52.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-06-03T19:05:06.000Z (12 months ago)
- Last Synced: 2025-03-26T04:34:04.675Z (about 2 months ago)
- Topics: instance-segmentation, object-detection
- Language: Python
- Homepage: https://arxiv.org/abs/2111.11057
- Size: 129 KB
- Stars: 75
- Watchers: 1
- Forks: 10
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Context Aggregation Network
[](https://arxiv.org/abs/2111.11057)
[](https://github.com/yeliudev/CATNet/blob/main/LICENSE)
This repository maintains the official implementation of the paper **Learning to Aggregate Multi-Scale Context for Instance Segmentation in Remote Sensing Images** by [Ye Liu](https://yeliu.dev/), [Huifang Li](https://faculty.whu.edu.cn/show.jsp?n=Huifang%20Li), [Chao Hu](https://orcid.org/0000-0001-6183-9051), [Shuang Luo](https://www.researchgate.net/profile/shuang-luo-6), [Yan Luo](https://www.researchgate.net/profile/Yan_Luo65), and [Chang Wen Chen](https://web.comp.polyu.edu.hk/chencw/), which has been accepted by [TNNLS](https://cis.ieee.org/publications/t-neural-networks-and-learning-systems).
## Installation
Please refer to the following environmental settings that we use. You may install these packages by yourself if you meet any problem during automatic installation.
- CUDA 11.8
- CUDNN 8.7.0.84
- Python 3.11.3
- PyTorch 2.0.1
- [MMEngine](https://github.com/open-mmlab/mmengine) 0.7.4
- [MMCV](https://github.com/open-mmlab/mmcv) 2.0.0
- [MMDetection](https://github.com/open-mmlab/mmdetection) 3.0.0
- [NNCore](https://github.com/yeliudev/nncore) 0.3.6
### Install from source
1. Clone the repository from GitHub.
```
git clone https://github.com/yeliudev/CATNet.git
cd CATNet
```
2. Install dependencies.
```
pip install -r requirements.txt
```
3. Set environment variable
```
export PYTHONPATH=$PWD:$PYTHONPATH
```
## Getting Started
### Download and prepare the datasets
1. Download the pre-processed datasets.
- [iSAID](https://huggingface.co/yeliudev/CATNet/resolve/main/datasets/isaid-9d62d4ad.zip)
- [DIOR](https://huggingface.co/yeliudev/CATNet/resolve/main/datasets/dior-b162132d.zip)
- [NWPU VHR-10](https://huggingface.co/yeliudev/CATNet/resolve/main/datasets/vhr-79ccc9f3.zip)
- [HRSID](https://huggingface.co/yeliudev/CATNet/resolve/main/datasets/hrsid-4e02052e.zip)
2. Prepare the files in the following structure.
```
CATNet
├── configs
├── datasets
├── models
├── tools
├── data
│ ├── dior
│ │ ├── Annotations
│ │ ├── ImageSets
│ │ └── JPEGImages
│ ├── hrsid
│ │ ├── annotations
│ │ └── images
│ ├── isaid
│ │ ├── train
│ │ ├── val
│ │ └── test
│ └── vhr
│ ├── annotations
│ └── images
├── README.md
├── setup.cfg
└── ···
```
### Train a model
Run the following command to train a model using a specified config.
```
mim train mmdet --gpus 4 --launcher pytorch
```
> If an `out-of-memory` error occurs on iSAID dataset, please uncomment [L22-L24](https://github.com/yeliudev/CATNet/blob/main/datasets/isaid.py#L22:L24) in the dataset code and try again. This will filter out a few images with more than 1,000 objects, largely reducing the memory cost.
### Test a model and evaluate results
Run the following command to test a model and evaluate results.
```
mim test mmdet --checkpoint --gpus 4 --launcher pytorch
```
## Model Zoo
We provide multiple pre-trained models here. All the models are trained using 4 NVIDIA A100 GPUs and are evaluated using the default metrics of the datasets.
Dataset
Model
Backbone
Schd
Aug
Performance
Download
BBox AP
Mask AP
iSAID
CAT Mask R-CNN
ResNet-50
3x
✗
45.1
37.2
model |
metrics
CAT Mask R-CNN
ResNet-50
3x
✓
47.7
39.2
model |
metrics
DIOR
CATNet
ResNet-50
3x
✗
74.0
—
model |
metrics
CATNet
ResNet-50
3x
✓
78.2
—
model |
metrics
CAT R-CNN
ResNet-50
3x
✗
75.8
—
model |
metrics
CAT R-CNN
ResNet-50
3x
✓
80.6
—
model |
metrics
NWPU
VHR-10
CAT Mask R-CNN
ResNet-50
6x
✗
71.0
69.3
model |
metrics
CAT Mask R-CNN
ResNet-50
6x
✓
72.4
70.7
model |
metrics
HRSID
CAT Mask R-CNN
ResNet-50
6x
✗
70.9
57.6
model |
metrics
CAT Mask R-CNN
ResNet-50
6x
✓
72.0
59.6
model |
metrics
## Citation
If you find this project useful for your research, please kindly cite our paper.
```bibtex
@article{liu2024learning,
title={Learning to Aggregate Multi-Scale Context for Instance Segmentation in Remote Sensing Images},
author={Liu, Ye and Li, Huifang and Hu, Chao and Luo, Shuang and Luo, Yan and Chen, Chang Wen},
journal={IEEE Transactions on Neural Networks and Learning Systems},
year={2024}
}
```