Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/TuSimple/simpledet
A Simple and Versatile Framework for Object Detection and Instance Recognition
https://github.com/TuSimple/simpledet
instance-segmentation mxnet object-detection
Last synced: 6 days ago
JSON representation
A Simple and Versatile Framework for Object Detection and Instance Recognition
- Host: GitHub
- URL: https://github.com/TuSimple/simpledet
- Owner: tusen-ai
- License: apache-2.0
- Created: 2019-01-29T08:28:55.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2021-09-23T02:57:09.000Z (about 3 years ago)
- Last Synced: 2024-11-01T12:36:01.603Z (12 days ago)
- Topics: instance-segmentation, mxnet, object-detection
- Language: Python
- Homepage:
- Size: 1.34 MB
- Stars: 3,087
- Watchers: 119
- Forks: 488
- Open Issues: 44
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- Awesome-MXNet - **simpledet**
README
## SimpleDet - A Simple and Versatile Framework for Object Detection and Instance Recognition
### Major Features
- FP16 training for memory saving and up to **2.5X** acceleration
- Highly scalable distributed training available **out of box**
- Full coverage of state-of-the-art models including FasterRCNN, MaskRCNN, CascadeRCNN, RetinaNet, [DCNv1/v2](./models/dcn), **[TridentNet](./models/tridentnet)**, **[NASFPN](./models/NASFPN)** , **[EfficientNet](./models/efficientnet)**, and **[Knowledge Distillation](./models/KD)**
- Extensive feature set including **large batch BN**, **loss synchronization**, **automatic BN fusion**, soft NMS, multi-scale train/test
- Modular design for coding-free exploration of new experiment settings
- Extensive documentations including [annotated config](./doc/fully_annotated_config.py), [Fintuning Guide](./doc/FINETUNE.md)
### Recent Updates
- Add RPN test (2019.05.28)
- Add [NASFPN](https://github.com/TuSimple/simpledet/tree/master/models/NASFPN) (2019.06.04)
- Add new ResNetV1b baselines from GluonCV (2019.06.07)
- Add Cascade R-CNN with FPN backbone (2019.06.11)
- Speed up FPN up to 70% (2019.06.16)
- Update [NASFPN](https://github.com/TuSimple/simpledet/tree/master/models/NASFPN) to include larger models (2019.07.01)
- Automatic BN fusion for fixed BN training, saving up to 50% GPU memory (2019.07.04)
- Speed up MaskRCNN by 80% (2019.07.23)
- Update MaskRCNN baselines (2019.07.25)
- Add EfficientNet and DCN (2019.08.06)
- Add python wheel for easy local installation (2019.08.20)
- Add FitNet based Knowledge Distill (2019.08.27)
- Add SE and train from scratch (2019.08.30)
- Add a lot of docs (2019.09.03)
- Add support for INT8 training(contributed by Xiaotao Chen & Jingqiu Zhou) (2019.10.24)
- Add support for FCOS(contributed by Zhen Wei) (2019.11)
- Add support for Mask Scoring RCNN(contributed by Zehui Chen) (2019.12)
- Add support for RepPoints(contributed by Bo Ke) (2020.02)
- Add support for FreeAnchor (2020.03)
- Add support for Feature Pyramid Grids & PAFPN (2020.06)
- Add support for CrowdHuman Dataset (2020.06)
- Add support for [Double Pred](https://arxiv.org/abs/2003.09163) (2020.06)
- Add support for [SEPC](https://github.com/TuSimple/simpledet/tree/master/models/sepc)(contributed by Qiaofei Li) (2020.07)
### Setup
#### All-in-one Script
We provide a [setup script](./scripts/setup.sh) for install simpledet and preppare the coco dataset. If you use this script, you can skip to the Quick Start.
#### Install
We provide a conda installation here for Debian/Ubuntu system. To use a pre-built docker or singularity images, please refer to [INSTALL.md](./doc/INSTALL.md) for more information.
```bash
# install dependency
sudo apt update && sudo apt install -y git wget make python3-dev libglib2.0-0 libsm6 libxext6 libxrender-dev unzip
# create conda env
conda create -n simpledet python=3.7
conda activate simpledet
# fetch CUDA environment
conda install cudatoolkit=10.1
# install python dependency
pip install 'matplotlib<3.1' opencv-python pytz
# download and intall pre-built wheel for CUDA 10.1
pip install https://1dv.aflat.top/mxnet_cu101-1.6.0b20191214-py2.py3-none-manylinux1_x86_64.whl
# install pycocotools
pip install 'git+https://github.com/RogerChern/cocoapi.git#subdirectory=PythonAPI'
# install mxnext, a wrapper around MXNet symbolic API
pip install 'git+https://github.com/RogerChern/mxnext#egg=mxnext'
# get simpledet
git clone https://github.com/tusimple/simpledet
cd simpledet
make
# test simpledet installation
mkdir -p experiments/faster_r50v1_fpn_1x
python detection_infer_speed.py --config config/faster_r50v1_fpn_1x.py --shape 800 1333
```
If the last line execute successfully, the average running speed of Faster R-CNN R-50 FPN will be reported. And you have successfuly setup SimpleDet. Now you can head up to the next section to prepare your dataset.
#### Preparing Data
We provide a step by step preparation for the COCO dataset below.
```bash
cd simpledet
# make data dir
mkdir -p data/coco/images data/src
# skip this if you have the zip files
wget -c http://images.cocodataset.org/zips/train2017.zip -O data/src/train2017.zip
wget -c http://images.cocodataset.org/zips/val2017.zip -O data/src/val2017.zip
wget -c http://images.cocodataset.org/zips/test2017.zip -O data/src/test2017.zip
wget -c http://images.cocodataset.org/annotations/annotations_trainval2017.zip -O data/src/annotations_trainval2017.zip
wget -c http://images.cocodataset.org/annotations/image_info_test2017.zip -O data/src/image_info_test2017.zip
unzip data/src/train2017.zip -d data/coco/images
unzip data/src/val2017.zip -d data/coco/images
unzip data/src/test2017.zip -d data/coco/images
unzip data/src/annotations_trainval2017.zip -d data/coco
unzip data/src/image_info_test2017.zip -d data/coco
python utils/create_coco_roidb.py --dataset coco --dataset-split train2017
python utils/create_coco_roidb.py --dataset coco --dataset-split val2017
python utils/create_coco_roidb.py --dataset coco --dataset-split test-dev2017
```
For other datasets or your own data, please check [DATASET.md](doc/DATASET.md) for more details.
#### Quick Start
```bash
# train
python detection_train.py --config config/faster_r50v1_fpn_1x.py
# test
python detection_test.py --config config/faster_r50v1_fpn_1x.py
```
#### Finetune
Please check [FINTUNE.md](doc/FINETUNE.md)
#### Model Zoo
Please refer to [MODEL_ZOO.md](./MODEL_ZOO.md) for available models
### Distributed Training
Please refer to [DISTRIBUTED.md](./doc/DISTRIBUTED.md)
### Project Organization
#### Code Structure
```
detection_train.py
detection_test.py
config/
detection_config.py
core/
detection_input.py
detection_metric.py
detection_module.py
models/
FPN/
tridentnet/
maskrcnn/
cascade_rcnn/
retinanet/
mxnext/
symbol/
builder.py
```
#### Config
Everything is configurable from the config file, all the changes should be **out of source**.
#### Experiments
One experiment is a directory in **experiments** folder with the same name as the config file.
> E.g. r50_fixbn_1x.py is the name of a config file
```
config/
r50_fixbn_1x.py
experiments/
r50_fixbn_1x/
checkpoint.params
log.txt
coco_minival2014_result.json
```
#### Models
The `models` directory contains SOTA models implemented in SimpletDet.
#### How is Faster R-CNN built
**Simpledet** supports many popular detection methods and here we take [**Faster R-CNN**](https://arxiv.org/abs/1506.01497) as a typical example to show how a detector is built.
- *Preprocessing*. The preprocessing methods of the detector is implemented through `DetectionAugmentation`.
- Image/bbox-related preprocessing, such as `Norm2DImage` and `Resize2DImageBbox`.
- Anchor generator `AnchorTarget2D`, which generates anchors and corresponding anchor targets for training RPN.
- *Network Structure*. The training and testing symbols of Faster-RCNN detector is defined in `FasterRcnn`. The key components are listed as follow:
- *Backbone*. `Backbone` provides interfaces to build backbone networks, *e.g.* ResNet and ResNext.
- *Neck*. `Neck` provides interfaces to build complementary feature extraction layers for backbone networks, *e.g.* `FPNNeck` builds Top-down pathway for [Feature Pyramid Network](https://arxiv.org/abs/1612.03144).
- *RPN head*. `RpnHead` aims to build classification and regression layers to generate proposal outputs for RPN. Meanwhile, it also provides interplace to generate sampled proposals for the subsequent R-CNN.
- *Roi Extractor*. `RoiExtractor` extracts features for each roi (proposal) based on the R-CNN features generated by `Backbone` and `Neck`.
- *Bounding Box Head*. `BboxHead` builds the R-CNN layers for proposal refinement.
#### How to build a custom detector
The flexibility of **simpledet** framework makes it easy to build different detectors. We take [**TridentNet**](https://arxiv.org/abs/1901.01892) as an example to demonstrate how to build a custom detector simply based on the Faster R-CNN framework.
- *Preprocessing*. The additional processing methods could be provided accordingly by inheriting from `DetectionAugmentation`.
- In TridentNet, a new `TridentAnchorTarget2D` is implemented to generate anchors for multiple branches and filter anchors for scale-aware training scheme.
- *Network Structure*. The new network structure could be constructed easily for a custom detector by modifying some required components as needed and
- For TridentNet, we build trident blocks in the `Backbone` according to the descriptions in the paper. We also provide a `TridentRpnHead` to generate filtered proposals in RPN to implement the scale-aware scheme. Other components are shared the same with original Faster-RCNN.
### Contributors
Yuntao Chen, Chenxia Han, Yanghao Li, Zehao Huang, Naiyan Wang, Xiaotao Chen, Jingqiu Zhou, Zhen Wei, Zehui Chen, Zhaoxiang Zhang, Bo Ke
### License and Citation
This project is release under the Apache 2.0 license for non-commercial usage. For commercial usage, please contact us for another license.
If you find our project helpful, please consider cite our tech report.
```
@article{JMLR:v20:19-205,
author = {Yuntao Chen and Chenxia Han and Yanghao Li and Zehao Huang and Yi Jiang and Naiyan Wang and Zhaoxiang Zhang},
title = {SimpleDet: A Simple and Versatile Distributed Framework for Object Detection and Instance Recognition},
journal = {Journal of Machine Learning Research},
year = {2019},
volume = {20},
number = {156},
pages = {1-8},
url = {http://jmlr.org/papers/v20/19-205.html}
}
```