https://github.com/lyuwenyu/RT-DETR

[CVPR 2024] Official RT-DETR (RTDETR paddle pytorch), Real-Time DEtection TRansformer, DETRs Beat YOLOs on Real-time Object Detection. 🔥 🔥 🔥
https://github.com/lyuwenyu/RT-DETR

rtdetr rtdetrv2

Last synced: 2 months ago
JSON representation

[CVPR 2024] Official RT-DETR (RTDETR paddle pytorch), Real-Time DEtection TRansformer, DETRs Beat YOLOs on Real-time Object Detection. 🔥 🔥 🔥

Host: GitHub
URL: https://github.com/lyuwenyu/RT-DETR
Owner: lyuwenyu
License: apache-2.0
Created: 2023-05-10T06:35:56.000Z (almost 2 years ago)
Default Branch: main
Last Pushed: 2024-11-28T07:51:25.000Z (5 months ago)
Last Synced: 2024-12-04T09:06:50.708Z (5 months ago)
Topics: rtdetr, rtdetrv2
Language: Python
Homepage:
Size: 603 KB
Stars: 2,726
Watchers: 25
Forks: 312
Open Issues: 313
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE

Awesome Lists containing this project

awesome-yolo-object-detection - RT-DETR | RT-DETRv2 - DETR?style=social"/> : "DETRs Beat YOLOs on Real-time Object Detection". (**[CVPR 2024](https://arxiv.org/abs/2304.08069)**). "RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer". (**[arXiv 2024](https://arxiv.org/abs/2407.17140)**). (Summary)

README

        English | [简体中文](README_cn.md)

RT-DETR: DETRs Beat YOLOs on Real-time Object Detection



    

    

        

    

    

        

    

    

        

    

    

        

    

    

        

    

    

        

    



---

This is the official implementation of papers 

- [DETRs Beat YOLOs on Real-time Object Detection](https://arxiv.org/abs/2304.08069)

- [RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer](https://arxiv.org/abs/2407.17140)

Fig





## 🚀 Updates

- \[2024.11.28\] Add torch tool for parameters and flops statistics. see [run_profile.py](./rtdetrv2_pytorch/tools/run_profile.py)

- \[2024.10.10\] Add sliced inference support for small object detecion. [#468](https://github.com/lyuwenyu/RT-DETR/pull/468)

- \[2024.09.23\] Add ✅[Regnet and DLA34](https://github.com/lyuwenyu/RT-DETR/tree/main/rtdetr_pytorch) for RTDETR.

- \[2024.08.27\] Add hubconf.py file to support torch hub.

- \[2024.08.22\] Improve the performance of ✅ [RT-DETRv2-S](./rtdetrv2_pytorch/) to 48.1 mAP (+1.6 compared to RT-DETR-R18).

- \[2024.07.24\] Release ✅ [RT-DETRv2](./rtdetrv2_pytorch/)!

- \[2024.02.27\] Our work has been accepted to CVPR 2024!

- \[2024.01.23\] Fix difference on data augmentation with paper in rtdetr_pytorch [#84](https://github.com/lyuwenyu/RT-DETR/commit/5dc64138e439247b4e707dd6cebfe19d8d77f5b1).

- \[2023.11.07\] Add pytorch ✅ *rtdetr_r34vd* for requests [#107](https://github.com/lyuwenyu/RT-DETR/issues/107), [#114](https://github.com/lyuwenyu/RT-DETR/issues/114).

- \[2023.11.05\] Upgrade the logic of `remap_mscoco_category` to facilitate training of custom datasets, see detils in [*Train custom data*](./rtdetr_pytorch/) part. [#81](https://github.com/lyuwenyu/RT-DETR/commit/95fc522fd7cf26c64ffd2ad0c622c392d29a9ebf).

- \[2023.10.23\] Add [*discussion for deployments*](https://github.com/lyuwenyu/RT-DETR/issues/95), supported onnxruntime, TensorRT, openVINO.

- \[2023.10.12\] Add tuning code for pytorch version, now you can tuning rtdetr based on pretrained weights.

- \[2023.09.19\] Upload ✅ [*pytorch weights*](https://github.com/lyuwenyu/RT-DETR/issues/42) convert from paddle version.

- \[2023.08.24] Release RT-DETR-R18 pretrained models on objects365. *49.2 mAP* and *217 FPS*.

- \[2023.08.22\] Upload ✅ [*rtdetr_pytorch*](./rtdetr_pytorch/) source code. Please enjoy it!

- \[2023.08.15\] Release RT-DETR-R101 pretrained models on objects365. *56.2 mAP* and *74 FPS*.

- \[2023.07.30\] Release RT-DETR-R50 pretrained models on objects365. *55.3 mAP* and *108 FPS*.

- \[2023.07.28\] Fix some bugs, and add some comments. [1](https://github.com/lyuwenyu/RT-DETR/pull/14), [2](https://github.com/lyuwenyu/RT-DETR/commit/3b5cbcf8ae3b907e6b8bb65498a6be7c6736eabc).

- \[2023.07.13\] Upload ✅ [*training logs on coco*](https://github.com/lyuwenyu/RT-DETR/issues/8).

- \[2023.05.17\] Release RT-DETR-R18, RT-DETR-R34, RT-DETR-R50-m（example for scaled).

- \[2023.04.17\] Release RT-DETR-R50, RT-DETR-R101, RT-DETR-L, RT-DETR-X.

## 📍 Implementations

- 🔥 RT-DETRv2

  - paddle: [code&weight](./rtdetrv2_paddle/)

  - pytorch: [code&weight](./rtdetrv2_pytorch/)

- 🔥 RT-DETR 

  - paddle: [code&weight](./rtdetr_paddle)

  - pytorch: [code&weight](./rtdetr_pytorch)

| Model | Input shape | Dataset | $AP^{val}$ | $AP^{val}_{50}$| Params(M) | FLOPs(G) | T4 TensorRT FP16(FPS)

|:---:|:---:| :---:|:---:|:---:|:---:|:---:|:---:|

| RT-DETR-R18 | 640 | COCO | 46.5 | 63.8 | 20 | 60 | 217 |

| RT-DETR-R34 | 640 | COCO | 48.9 | 66.8 | 31 | 92 | 161 |

| RT-DETR-R50-m | 640 | COCO | 51.3 | 69.6 | 36 | 100 | 145 |

| RT-DETR-R50 |  640 | COCO | 53.1 | 71.3 | 42 | 136 | 108 |

| RT-DETR-R101 | 640 | COCO | 54.3 | 72.7 | 76 | 259 | 74 |

| RT-DETR-HGNetv2-L | 640 | COCO | 53.0 | 71.6 | 32 | 110 | 114 |

| RT-DETR-HGNetv2-X | 640 | COCO | 54.8 | 73.1 | 67 | 234 | 74 |

| RT-DETR-R18 | 640 | COCO + Objects365 | **49.2** | **66.6** | 20 | 60 | **217** |

| RT-DETR-R50 | 640 | COCO + Objects365 | **55.3** | **73.4** | 42 | 136 | **108** |

| RT-DETR-R101 | 640 | COCO + Objects365 | **56.2** | **74.6** | 76 | 259 | **74** |

**RT-DETRv2-S** | 640 | COCO  | **48.1** (+1.6) | **65.1** | 20 | 60 | 217 |

**RT-DETRv2-M**^{*^{| 640 | COCO  | **49.9** (+1.0) | **67.5** | 31 | 92 | 161 |

**RT-DETRv2-M** | 640 | COCO | **51.9** (+0.6) | **69.9** | 36 | 100 | 145 |

**RT-DETRv2-L** | 640 | COCO | **53.4** (+0.3) | **71.6** | 42 | 136 | 108 |

**RT-DETRv2-X** | 640 | COCO | 54.3 | **72.8** (+0.1)  | 76 | 259| 74 |}}

**Notes:**

- `COCO + Objects365` in the table means finetuned model on COCO using pretrained weights trained on Objects365.

## 🦄 Performance

### 🏕️ Complex Scenarios



  



### 🌋 Difficult Conditions



  



## Citation

If you use `RT-DETR` or `RTDETRv2` in your work, please use the following BibTeX entries:

```

@misc{lv2023detrs,

      title={DETRs Beat YOLOs on Real-time Object Detection},

      author={Yian Zhao and Wenyu Lv and Shangliang Xu and Jinman Wei and Guanzhong Wang and Qingqing Dang and Yi Liu and Jie Chen},

      year={2023},

      eprint={2304.08069},

      archivePrefix={arXiv},

      primaryClass={cs.CV}

}

@misc{lv2024rtdetrv2improvedbaselinebagoffreebies,

      title={RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer}, 

      author={Wenyu Lv and Yian Zhao and Qinyao Chang and Kui Huang and Guanzhong Wang and Yi Liu},

      year={2024},

      eprint={2407.17140},

      archivePrefix={arXiv},

      primaryClass={cs.CV},

      url={https://arxiv.org/abs/2407.17140}, 

}

```

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/lyuwenyu/RT-DETR

Awesome Lists containing this project

README

RT-DETR: DETRs Beat YOLOs on Real-time Object Detection