{"id":16270222,"url":"https://github.com/grimoire/mmdetection-to-tensorrt","last_synced_at":"2025-04-04T21:08:27.771Z","repository":{"id":39652972,"uuid":"279044554","full_name":"grimoire/mmdetection-to-tensorrt","owner":"grimoire","description":"convert mmdetection model to tensorrt, support fp16, int8, batch input, dynamic shape etc.","archived":false,"fork":false,"pushed_at":"2024-02-18T10:06:42.000Z","size":426,"stargazers_count":586,"open_issues_count":65,"forks_count":85,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-10-11T18:09:58.382Z","etag":null,"topics":["cascade-rcnn","faster-rcnn","inference","mmdetection","object-detection","retinanet","ssd","tensorrt","yolov3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/grimoire.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-12T10:50:30.000Z","updated_at":"2024-09-25T13:37:12.000Z","dependencies_parsed_at":"2024-02-18T08:25:02.593Z","dependency_job_id":"0aab3f5c-fc9d-42a1-aecf-3c619524cd9f","html_url":"https://github.com/grimoire/mmdetection-to-tensorrt","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grimoire%2Fmmdetection-to-tensorrt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grimoire%2Fmmdetection-to-tensorrt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grimoire%2Fmmdetection-to-tensorrt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grimoire%2Fmmdetection-to-tensorrt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/grimoire","download_url":"https://codeload.github.com/grimoire/mmdetection-to-tensorrt/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247249526,"owners_count":20908212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cascade-rcnn","faster-rcnn","inference","mmdetection","object-detection","retinanet","ssd","tensorrt","yolov3"],"created_at":"2024-10-10T18:09:53.704Z","updated_at":"2025-04-04T21:08:27.750Z","avatar_url":"https://github.com/grimoire.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MMDet to TensorRT\n\n\u003e [!NOTE]\n\u003e\n\u003e The main branch is used to support model conversion of MMDetection\u003e=3.0.\n\u003e If you want to convert model on older MMDetection, Please switch to branch:\n\u003e - [mmdet2trt=v0.5.0](https://github.com/grimoire/mmdetection-to-tensorrt/tree/v0.5.0)\n\u003e - [torch2trt_dynamic=v0.5.0](https://github.com/grimoire/torch2trt_dynamic/tree/v0.5.0)\n\u003e - [amirstan_plugin=v0.5.0](https://github.com/grimoire/amirstan_plugin/tree/v0.5.0).\n\n## News\n\n- 2024.02: Support MMDetection\u003e=3.0\n\n## Introduction\n\nThis project aims to support End2End deployment of models in MMDetection with TensorRT.\n\nMask support is **experiment**.\n\nFeatures:\n\n- fp16\n- int8(experiment)\n- batched input\n- dynamic input shape\n- combination of different modules\n- DeepStream\n\n## Requirement\n\n- install MMDetection:\n\n    ```bash\n    pip install openmim\n    mim install mmdet==3.3.0\n    ```\n\n- install [torch2trt_dynamic](https://github.com/grimoire/torch2trt_dynamic):\n\n    ```bash\n    git clone https://github.com/grimoire/torch2trt_dynamic.git torch2trt_dynamic\n    cd torch2trt_dynamic\n    pip install -e .\n    ```\n\n- install [amirstan_plugin](https://github.com/grimoire/amirstan_plugin):\n  - Install tensorrt: [TensorRT](https://developer.nvidia.com/tensorrt)\n  - clone repo and build plugin\n\n    ```bash\n    git clone --depth=1 https://github.com/grimoire/amirstan_plugin.git\n    cd amirstan_plugin\n    git submodule update --init --progress --depth=1\n    mkdir build\n    cd build\n    cmake -DTENSORRT_DIR=${TENSORRT_DIR} ..\n    make -j10\n    ```\n\n    \u003e [!NOTE]\n    \u003e\n    \u003e   **DON'T FORGET** setting the environment variable(in `~/.bashrc`):\n    \u003e\n    \u003e ```bash\n    \u003e export AMIRSTAN_LIBRARY_PATH=${amirstan_plugin_root}/build/lib\n    \u003e ```\n\n## Installation\n\n### Host\n\n```bash\ngit clone https://github.com/grimoire/mmdetection-to-tensorrt.git\ncd mmdetection-to-tensorrt\npip install -e .\n```\n\n### Docker\n\nBuild docker image\n\n```bash\nsudo docker build -t mmdet2trt_docker:v1.0 docker/\n```\n\nRun (will show the help for the CLI entrypoint)\n\n```bash\nsudo docker run --gpus all -it --rm -v ${your_data_path}:${bind_path} mmdet2trt_docker:v1.0\n```\n\nOr if you want to open a terminal inside de container:\n\n```bash\nsudo docker run --gpus all -it --rm -v ${your_data_path}:${bind_path} --entrypoint bash mmdet2trt_docker:v1.0\n```\n\nExample conversion:\n\n```bash\nsudo docker run --gpus all -it --rm -v ${your_data_path}:${bind_path} mmdet2trt_docker:v1.0 ${bind_path}/config.py ${bind_path}/checkpoint.pth ${bind_path}/output.trt\n```\n\n## Usage\n\nCreate a TensorRT model from mmdet model.\ndetail can be found in [getting_started.md](./docs/getting_started.md)\n\n### CLI\n\n```bash\n# conversion might take few minutes.\nmmdet2trt ${CONFIG_PATH} ${CHECKPOINT_PATH} ${OUTPUT_PATH}\n```\n\nRun mmdet2trt -h for help on optional arguments.\n\n### Python\n\n```python\nshape_ranges=dict(\n    x=dict(\n        min=[1,3,320,320],\n        opt=[1,3,800,1344],\n        max=[1,3,1344,1344],\n    )\n)\ntrt_model = mmdet2trt(cfg_path,\n                      weight_path,\n                      shape_ranges=shape_ranges,\n                      fp16_mode=True)\n\n# save converted model\ntorch.save(trt_model.state_dict(), save_model_path)\n\n# save engine if you want to use it in c++ api\nwith open(save_engine_path, mode='wb') as f:\n    f.write(trt_model.state_dict()['engine'])\n```\n\n\u003e [!NOTE]\n\u003e\n\u003e The input of the engine is the tensor **after preprocess**.\n\u003e The output of the engine is `num_dets, bboxes, scores, class_ids`. if you enable the `enable_mask` flag, there will be another output `mask`.\n\u003e The bboxes output of the engine did not divided by `scale_factor`.\n\nhow to perform inference with the converted model.\n\n```python\nfrom mmdet.apis import inference_detector\nfrom mmdet2trt.apis import create_wrap_detector\n\n# create wrap detector\ntrt_detector = create_wrap_detector(trt_model, cfg_path, device_id)\n\n# result share same format as mmdetection\nresult = inference_detector(trt_detector, image_path)\n```\n\nTry demo in `demo/inference.py`, or `demo/cpp` if you want to do inference with c++ api.\n\nRead [getting_started.md](./docs/getting_started.md) for more details.\n\n## How does it works?\n\nMost other project use pytorch=\u003eONNX=\u003etensorRT route, This repo convert pytorch=\u003etensorRT directly, avoid unnecessary ONNX IR.\nRead [how-does-it-work](https://github.com/NVIDIA-AI-IOT/torch2trt#how-does-it-work) for detail.\n\n## Support Model/Module\n\n\u003e [!NOTE]\n\u003e\n\u003e Some models have only been tested on MMDet\u003c3.0. If you found any failed model,\n\u003e Please report in the issue.\n\n- [x] Faster R-CNN\n- [x] Cascade R-CNN\n- [x] Double-Head R-CNN\n- [x] Group Normalization\n- [x] Weight Standardization\n- [x] DCN\n- [x] SSD\n- [x] RetinaNet\n- [x] Libra R-CNN\n- [x] FCOS\n- [x] Fovea\n- [x] CARAFE\n- [x] FreeAnchor\n- [x] RepPoints\n- [x] NAS-FPN\n- [x] ATSS\n- [x] PAFPN\n- [x] FSAF\n- [x] GCNet\n- [x] Guided Anchoring\n- [x] Generalized Attention\n- [x] Dynamic R-CNN\n- [x] Hybrid Task Cascade\n- [x] DetectoRS\n- [x] Side-Aware Boundary Localization\n- [x] YOLOv3\n- [x] PAA\n- [ ] CornerNet(WIP)\n- [x] Generalized Focal Loss\n- [x] Grid RCNN\n- [x] VFNet\n- [x] GROIE\n- [x] Mask R-CNN(experiment)\n- [x] Cascade Mask R-CNN(experiment)\n- [x] Cascade RPN\n- [x] DETR\n- [x] YOLOX\n\nTested on:\n\n- torch=2.2.0\n- tensorrt=8.6.1\n- mmdetection=3.3.0\n- cuda=11.7\n\n## FAQ\n\nread [this page](./docs/FAQ.md) if you meet any problem.\n\n## License\n\nThis project is released under the [Apache 2.0 license](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrimoire%2Fmmdetection-to-tensorrt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgrimoire%2Fmmdetection-to-tensorrt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrimoire%2Fmmdetection-to-tensorrt/lists"}