{"id":13400110,"url":"https://github.com/RangiLyu/nanodet","last_synced_at":"2025-03-14T04:32:34.986Z","repository":{"id":38691962,"uuid":"305316768","full_name":"RangiLyu/nanodet","owner":"RangiLyu","description":"NanoDet-Plus⚡Super fast and lightweight anchor-free object detection model. 🔥Only 980 KB(int8) / 1.8MB (fp16) and run 97FPS on cellphone🔥","archived":false,"fork":false,"pushed_at":"2024-08-08T02:52:55.000Z","size":5420,"stargazers_count":5740,"open_issues_count":238,"forks_count":1040,"subscribers_count":68,"default_branch":"main","last_synced_at":"2024-10-29T15:05:00.782Z","etag":null,"topics":["anchor-free","android","deep-learning","deep-neural-networks","efficientnet","mnn","model-zoo","nanodet","nanodet-plus","ncnn","object-detection","openvino","pytorch","repvgg","shufflenet"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RangiLyu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-19T08:36:25.000Z","updated_at":"2024-10-29T07:56:25.000Z","dependencies_parsed_at":"2024-06-18T21:17:42.057Z","dependency_job_id":"f5a1b7c8-df8c-46bb-b681-53ea6506051d","html_url":"https://github.com/RangiLyu/nanodet","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RangiLyu%2Fnanodet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RangiLyu%2Fnanodet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RangiLyu%2Fnanodet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RangiLyu%2Fnanodet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RangiLyu","download_url":"https://codeload.github.com/RangiLyu/nanodet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243125547,"owners_count":20240276,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anchor-free","android","deep-learning","deep-neural-networks","efficientnet","mnn","model-zoo","nanodet","nanodet-plus","ncnn","object-detection","openvino","pytorch","repvgg","shufflenet"],"created_at":"2024-07-30T19:00:48.431Z","updated_at":"2025-03-14T04:32:34.977Z","avatar_url":"https://github.com/RangiLyu.png","language":"Python","funding_links":[],"categories":["Machine Learning","🎯 Object Detection","⚡ Efficient Mobile Models","Python","对象检测、分割","Frameworks","Application projects","📦 Legacy \u0026 Inactive Projects"],"sub_categories":["🌟 NanoDet (Lightweight Detection)","🎯 Efficient Object Detection","网络服务_其他","Detection"],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003cimg src=\"docs/imgs/Title.jpg\" /\u003e\n\n# NanoDet-Plus\n**Super fast and high accuracy lightweight anchor-free object detection model. Real-time on mobile devices.**\n\n\n[![CI testing](https://img.shields.io/github/checks-status/RangiLyu/nanodet/main?label=CI\u0026style=flat)](https://img.shields.io/github/checks-status/RangiLyu/nanodet/main?label=CI\u0026style=flat)\n![Codecov](https://img.shields.io/codecov/c/github/RangiLyu/nanodet?color=hotpink)\n[![GitHub license](https://img.shields.io/github/license/RangiLyu/nanodet?color=turquoise\u0026style=flat)](https://github.com/RangiLyu/nanodet/blob/main/LICENSE)\n[![Github downloads](https://img.shields.io/github/downloads/RangiLyu/nanodet/total?color=orange\u0026label=downloads\u0026logo=github\u0026logoColor=lightgrey\u0026style=flat)](https://img.shields.io/github/downloads/RangiLyu/nanodet/total?color=yellow\u0026label=Downloads\u0026logo=github\u0026logoColor=lightgrey\u0026style=flat)\n[![GitHub release (latest by date)](https://img.shields.io/github/v/release/RangiLyu/nanodet?style=flat)](https://img.shields.io/github/v/release/RangiLyu/nanodet?style=flat)\n\n\u003c/div\u003e\n\n* ⚡Super lightweight: Model file is only 980KB(INT8) or 1.8MB(FP16).\n* ⚡Super fast: 97fps(10.23ms) on mobile ARM CPU.\n* 👍High accuracy: Up to **34.3 mAP\u003csup\u003eval\u003c/sup\u003e@0.5:0.95** and still realtime on CPU.\n* 🤗Training friendly:  Much lower GPU memory cost than other models. Batch-size=80 is available on GTX1060 6G.\n* 😎Easy to deploy: Support various backends including **ncnn, MNN and OpenVINO**. Also provide **Android demo** based on ncnn inference framework.\n\n****\n\n# Introduction\n\n\n![](docs/imgs/nanodet-plus-arch.png)\n\nNanoDet is a FCOS-style one-stage anchor-free object detection model which using [Generalized Focal Loss](https://arxiv.org/pdf/2006.04388.pdf) as classification and regression loss.\n\nIn NanoDet-Plus, we propose a novel label assignment strategy with a simple **assign guidance module (AGM)** and a **dynamic soft label assigner (DSLA)** to solve the optimal label assignment problem in lightweight model training. We also introduce a light feature pyramid called Ghost-PAN to enhance multi-layer feature fusion. These improvements boost previous NanoDet's detection accuracy by **7 mAP** on COCO dataset.\n\n[NanoDet-Plus 知乎中文介绍](https://zhuanlan.zhihu.com/p/449912627)\n\n[NanoDet 知乎中文介绍](https://zhuanlan.zhihu.com/p/306530300)\n\nQQ交流群：908606542 (答案：炼丹)\n\n****\n## Benchmarks\n\nModel          |Resolution| mAP\u003csup\u003eval\u003cbr\u003e0.5:0.95 |CPU Latency\u003csup\u003e\u003cbr\u003e(i7-8700) |ARM Latency\u003csup\u003e\u003cbr\u003e(4xA76) | FLOPS      |   Params  | Model Size\n:-------------:|:--------:|:-------:|:--------------------:|:--------------------:|:----------:|:---------:|:-------:\nNanoDet-m      | 320*320 |   20.6   | **4.98ms**           | **10.23ms**          | **0.72G**  | **0.95M** | **1.8MB(FP16)** \u0026#124; **980KB(INT8)**\n**NanoDet-Plus-m** | 320*320 | **27.0** | **5.25ms**       | **11.97ms**          | **0.9G**   | **1.17M** | **2.3MB(FP16)** \u0026#124; **1.2MB(INT8)**\n**NanoDet-Plus-m** | 416*416 | **30.4** | **8.32ms**       | **19.77ms**          | **1.52G**  | **1.17M** | **2.3MB(FP16)** \u0026#124; **1.2MB(INT8)**\n**NanoDet-Plus-m-1.5x** | 320*320 | **29.9** | **7.21ms**  | **15.90ms**          | **1.75G**  | **2.44M** | **4.7MB(FP16)** \u0026#124; **2.3MB(INT8)**\n**NanoDet-Plus-m-1.5x** | 416*416 | **34.1** | **11.50ms** | **25.49ms**          | **2.97G**   | **2.44M** | **4.7MB(FP16)** \u0026#124; **2.3MB(INT8)**\nYOLOv3-Tiny    | 416*416 |   16.6   | -                    | 37.6ms               | 5.62G      | 8.86M     |   33.7MB\nYOLOv4-Tiny    | 416*416 |   21.7   | -                    | 32.81ms              | 6.96G      | 6.06M     |   23.0MB\nYOLOX-Nano     | 416*416 |   25.8   | -                    | 23.08ms              | 1.08G      | 0.91M     |   1.8MB(FP16)\nYOLOv5-n       | 640*640 |   28.4   | -                    | 44.39ms              | 4.5G       | 1.9M      |   3.8MB(FP16)\nFBNetV5        | 320*640 |   30.4   | -                    | -                    | 1.8G       | -         |   -\nMobileDet      | 320*320 |   25.6   | -                    | -                    | 0.9G       | -         |   -\n\n***Download pre-trained models and find more models in [Model Zoo](#model-zoo) or in [Release Files](https://github.com/RangiLyu/nanodet/releases)***\n\n\u003cdetails\u003e\n    \u003csummary\u003eNotes (click to expand)\u003c/summary\u003e\n\n* ARM Performance is measured on Kirin 980(4xA76+4xA55) ARM CPU based on ncnn. You can test latency on your phone with [ncnn_android_benchmark](https://github.com/nihui/ncnn-android-benchmark).\n\n* Intel CPU Performance is measured Intel Core-i7-8700 based on OpenVINO.\n\n* NanoDet mAP(0.5:0.95) is validated on COCO val2017 dataset with no testing time augmentation.\n\n* YOLOv3\u0026YOLOv4 mAP refers from [Scaled-YOLOv4: Scaling Cross Stage Partial Network](https://arxiv.org/abs/2011.08036).\n\n\u003c/details\u003e\n\n****\n\n## NEWS!!!\n\n* [2023.01.20] Upgrade to [pytorch-lightning-1.9](https://github.com/Lightning-AI/lightning/releases/tag/1.9.0). The minimum PyTorch version is upgraded to 1.10. Support FP16 training(Thanks @crisp-snakey). Support ignore label(Thanks @zero0kiriyu).\n\n* [2022.08.26] Upgrade to [pytorch-lightning-1.7](https://lightning.ai/). The minimum PyTorch version is upgraded to 1.9. To use previous version of PyTorch, please install [NanoDet \u003c= v1.0.0-alpha-1](https://github.com/RangiLyu/nanodet/tags)\n\n* [2021.12.25] **NanoDet-Plus** release! Adding **AGM**(Assign Guidance Module) \u0026 **DSLA**(Dynamic Soft Label Assigner) to improve **7 mAP** with only a little cost.\n\nFind more update notes in [Update notes](docs/update.md).\n## Demo\n\n### Android demo\n\n![android_demo](docs/imgs/Android_demo.jpg)\n\nAndroid demo project is in ***demo_android_ncnn*** folder. Please refer to [Android demo guide](demo_android_ncnn/README.md).\n\nHere is a better implementation 👉 [ncnn-android-nanodet](https://github.com/nihui/ncnn-android-nanodet)\n\n### NCNN C++ demo\n\nC++ demo based on [ncnn](https://github.com/Tencent/ncnn) is in ***demo_ncnn*** folder. Please refer to [Cpp demo guide](demo_ncnn/README.md).\n\n### MNN demo\n\nInference using [Alibaba's MNN framework](https://github.com/alibaba/MNN) is in ***demo_mnn*** folder. Please refer to [MNN demo guide](demo_mnn/README.md).\n\n### OpenVINO demo\n\nInference using [OpenVINO](https://01.org/openvinotoolkit) is in ***demo_openvino*** folder. Please refer to [OpenVINO demo guide](demo_openvino/README.md).\n\n### Web browser demo\n\nhttps://nihui.github.io/ncnn-webassembly-nanodet/\n\n### Pytorch demo\n\nFirst, install requirements and setup NanoDet following installation guide. Then download COCO pretrain weight from here\n\n👉[COCO pretrain checkpoint](https://github.com/RangiLyu/nanodet/releases/download/v1.0.0-alpha-1/nanodet-plus-m_416_checkpoint.ckpt)\n\nThe pre-trained weight was trained by the config `config/nanodet-plus-m_416.yml`.\n\n* Inference images\n\n```bash\npython demo/demo.py image --config CONFIG_PATH --model MODEL_PATH --path IMAGE_PATH\n```\n\n* Inference video\n\n```bash\npython demo/demo.py video --config CONFIG_PATH --model MODEL_PATH --path VIDEO_PATH\n```\n\n* Inference webcam\n\n```bash\npython demo/demo.py webcam --config CONFIG_PATH --model MODEL_PATH --camid YOUR_CAMERA_ID\n```\n\nBesides, We provide a notebook [here](./demo/demo-inference-with-pytorch.ipynb) to demonstrate how to make it work with PyTorch.\n\n****\n\n## Install\n\n### Requirements\n\n* Linux or MacOS\n* CUDA \u003e= 10.2\n* Python \u003e= 3.7\n* Pytorch \u003e= 1.10.0, \u003c2.0.0\n\n### Step\n\n1. Create a conda virtual environment and then activate it.\n\n```shell script\n conda create -n nanodet python=3.8 -y\n conda activate nanodet\n```\n\n2. Install pytorch\n\n```shell script\nconda install pytorch torchvision cudatoolkit=11.1 -c pytorch -c conda-forge\n```\n\n3. Clone this repository\n\n```shell script\ngit clone https://github.com/RangiLyu/nanodet.git\ncd nanodet\n```\n\n4. Install requirements\n\n```shell script\npip install -r requirements.txt\n```\n\n5. Setup NanoDet\n```shell script\npython setup.py develop\n```\n\n****\n\n## Model Zoo\n\nNanoDet supports variety of backbones. Go to the [***config*** folder](config/) to see the sample training config files.\n\nModel                 | Backbone           |Resolution|COCO mAP| FLOPS |Params | Pre-train weight |\n:--------------------:|:------------------:|:--------:|:------:|:-----:|:-----:|:-----:|\nNanoDet-m             | ShuffleNetV2 1.0x  | 320*320  |  20.6  | 0.72G | 0.95M | [Download](https://drive.google.com/file/d/1ZkYucuLusJrCb_i63Lid0kYyyLvEiGN3/view?usp=sharing) |\nNanoDet-Plus-m-320 (***NEW***)     | ShuffleNetV2 1.0x | 320*320  |  27.0  | 0.9G  | 1.17M | [Weight](https://drive.google.com/file/d/1Dq0cTIdJDUhQxJe45z6rWncbZmOyh1Tv/view?usp=sharing) \u0026#124; [Checkpoint](https://drive.google.com/file/d/1YvuEhahlgqxIhJu7bsL-fhaqubKcCWQc/view?usp=sharing)\nNanoDet-Plus-m-416 (***NEW***)     | ShuffleNetV2 1.0x | 416*416  |  30.4  | 1.52G | 1.17M | [Weight](https://drive.google.com/file/d/1FN3WK3FLjBm7oCqiwUcD3m3MjfqxuzXe/view?usp=sharing) \u0026#124; [Checkpoint](https://drive.google.com/file/d/1gFjyrl7O8p5APr1ZOtWEm3tQNN35zi_W/view?usp=sharing)\nNanoDet-Plus-m-1.5x-320 (***NEW***)| ShuffleNetV2 1.5x | 320*320  |  29.9  | 1.75G | 2.44M | [Weight](https://drive.google.com/file/d/1Xdlgu5lxiS3w6ER7GE1mZpY663wmpcyY/view?usp=sharing) \u0026#124; [Checkpoint](https://drive.google.com/file/d/1qXR6t3TBMXlz6GlTU3fxiLA-eueYoGrW/view?usp=sharing)\nNanoDet-Plus-m-1.5x-416 (***NEW***)| ShuffleNetV2 1.5x | 416*416  |  34.1  | 2.97G | 2.44M | [Weight](https://drive.google.com/file/d/16FJJJgUt5VrSKG7RM_ImdKKzhJ-Mu45I/view?usp=sharing) \u0026#124; [Checkpoint](https://drive.google.com/file/d/17sdAUydlEXCrHMsxlDPLj5cGb-8-mmY6/view?usp=sharing)\n\n\n*Notice*: The difference between `Weight` and `Checkpoint` is the weight only provide params in inference time, but the checkpoint contains training time params.\n\n\nLegacy Model Zoo\n\nModel                 | Backbone           |Resolution|COCO mAP| FLOPS |Params | Pre-train weight |\n:--------------------:|:------------------:|:--------:|:------:|:-----:|:-----:|:-----:|\nNanoDet-m-416         | ShuffleNetV2 1.0x  | 416*416  |  23.5  |  1.2G | 0.95M | [Download](https://drive.google.com/file/d/1jY-Um2VDDEhuVhluP9lE70rG83eXQYhV/view?usp=sharing)|\nNanoDet-m-1.5x        | ShuffleNetV2 1.5x  | 320*320  |  23.5  | 1.44G | 2.08M | [Download](https://drive.google.com/file/d/1_n1cAWy622LV8wbUnXImtcvcUVPOhYrW/view?usp=sharing) |\nNanoDet-m-1.5x-416    | ShuffleNetV2 1.5x  | 416*416  |  26.8  | 2.42G | 2.08M | [Download](https://drive.google.com/file/d/1CCYgwX3LWfN7hukcomhEhGWN-qcC3Tv4/view?usp=sharing)|\nNanoDet-m-0.5x        | ShuffleNetV2 0.5x  | 320*320  |  13.5  |  0.3G | 0.28M | [Download](https://drive.google.com/file/d/1rMHkD30jacjRpslmQja5jls86xd0YssR/view?usp=sharing) |\nNanoDet-t             | ShuffleNetV2 1.0x  | 320*320  |  21.7  | 0.96G | 1.36M | [Download](https://drive.google.com/file/d/1TqRGZeOKVCb98ehTaE0gJEuND6jxwaqN/view?usp=sharing) |\nNanoDet-g             | Custom CSP Net     | 416*416  |  22.9  |  4.2G | 3.81M | [Download](https://drive.google.com/file/d/1f2lH7Ae1AY04g20zTZY7JS_dKKP37hvE/view?usp=sharing)|\nNanoDet-EfficientLite | EfficientNet-Lite0 | 320*320  |  24.7  | 1.72G | 3.11M | [Download](https://drive.google.com/file/d/1Dj1nBFc78GHDI9Wn8b3X4MTiIV2el8qP/view?usp=sharing)|\nNanoDet-EfficientLite | EfficientNet-Lite1 | 416*416  |  30.3  | 4.06G | 4.01M | [Download](https://drive.google.com/file/d/1ernkb_XhnKMPdCBBtUEdwxIIBF6UVnXq/view?usp=sharing) |\nNanoDet-EfficientLite | EfficientNet-Lite2 | 512*512  |  32.6  | 7.12G | 4.71M | [Download](https://drive.google.com/file/d/11V20AxXe6bTHyw3aMkgsZVzLOB31seoc/view?usp=sharing) |\nNanoDet-RepVGG        | RepVGG-A0          | 416*416  |  27.8  | 11.3G | 6.75M | [Download](https://drive.google.com/file/d/1nWZZ1qXb1HuIXwPSYpEyFHHqX05GaFer/view?usp=sharing) |\n\n\n****\n\n## How to Train\n\n1. **Prepare dataset**\n\n    If your dataset annotations are pascal voc xml format, refer to [config/nanodet_custom_xml_dataset.yml](config/nanodet_custom_xml_dataset.yml)\n\n    Otherwise, if your dataset annotations are YOLO format ([Darknet TXT](https://github.com/AlexeyAB/Yolo_mark/issues/60#issuecomment-401854885)), refer to [config/nanodet-plus-m_416-yolo.yml](config/nanodet-plus-m_416-yolo.yml)\n\n    Or convert your dataset annotations to MS COCO format[(COCO annotation format details)](https://cocodataset.org/#format-data).\n\n2. **Prepare config file**\n\n    Copy and modify an example yml config file in config/ folder.\n\n    Change ***save_dir*** to where you want to save model.\n\n    Change ***num_classes*** in ***model-\u003earch-\u003ehead***.\n\n    Change image path and annotation path in both ***data-\u003etrain*** and ***data-\u003eval***.\n\n    Set gpu ids, num workers and batch size in ***device*** to fit your device.\n\n    Set ***total_epochs***, ***lr*** and ***lr_schedule*** according to your dataset and batchsize.\n\n    If you want to modify network, data augmentation or other things, please refer to [Config File Detail](docs/config_file_detail.md)\n\n3. **Start training**\n\n   NanoDet is now using [pytorch lightning](https://github.com/PyTorchLightning/pytorch-lightning) for training.\n\n   For both single-GPU or multiple-GPUs, run:\n\n   ```shell script\n   python tools/train.py CONFIG_FILE_PATH\n   ```\n\n4. **Visualize Logs**\n\n    TensorBoard logs are saved in `save_dir` which you set in config file.\n\n    To visualize tensorboard logs, run:\n\n    ```shell script\n    cd \u003cYOUR_SAVE_DIR\u003e\n    tensorboard --logdir ./\n    ```\n\n****\n\n## How to Deploy\n\nNanoDet provide multi-backend C++ demo including ncnn, OpenVINO and MNN.\nThere is also an Android demo based on ncnn library.\n\n### Export model to ONNX\n\nTo convert NanoDet pytorch model to ncnn, you can choose this way: pytorch-\u003eonnx-\u003encnn\n\nTo export onnx model, run `tools/export_onnx.py`.\n\n```shell script\npython tools/export_onnx.py --cfg_path ${CONFIG_PATH} --model_path ${PYTORCH_MODEL_PATH}\n```\n\n### Run NanoDet in C++ with inference libraries\n\n### ncnn\n\nPlease refer to [demo_ncnn](demo_ncnn/README.md).\n\n### OpenVINO\n\nPlease refer to [demo_openvino](demo_openvino/README.md).\n\n### MNN\n\nPlease refer to [demo_mnn](demo_mnn/README.md).\n\n\n### Run NanoDet on Android\n\nPlease refer to [android_demo](demo_android_ncnn/README.md).\n\n****\n\n## Citation\n\nIf you find this project useful in your research, please consider cite:\n\n```BibTeX\n@misc{=nanodet,\n    title={NanoDet-Plus: Super fast and high accuracy lightweight anchor-free object detection model.},\n    author={RangiLyu},\n    howpublished = {\\url{https://github.com/RangiLyu/nanodet}},\n    year={2021}\n}\n```\n\n****\n\n## Thanks\n\nhttps://github.com/Tencent/ncnn\n\nhttps://github.com/open-mmlab/mmdetection\n\nhttps://github.com/implus/GFocal\n\nhttps://github.com/cmdbug/YOLOv5_NCNN\n\nhttps://github.com/rbgirshick/yacs\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRangiLyu%2Fnanodet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FRangiLyu%2Fnanodet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRangiLyu%2Fnanodet/lists"}