{"id":13415321,"url":"https://github.com/xingyizhou/CenterNet","last_synced_at":"2025-03-14T22:33:19.480Z","repository":{"id":37431225,"uuid":"181746604","full_name":"xingyizhou/CenterNet","owner":"xingyizhou","description":"Object detection, 3D detection, and pose estimation using center point detection: ","archived":false,"fork":false,"pushed_at":"2023-03-02T14:23:04.000Z","size":6412,"stargazers_count":7370,"open_issues_count":454,"forks_count":1927,"subscribers_count":112,"default_branch":"master","last_synced_at":"2025-03-11T10:09:01.454Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xingyizhou.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-04-16T18:39:13.000Z","updated_at":"2025-03-10T12:08:50.000Z","dependencies_parsed_at":"2023-10-21T01:00:42.650Z","dependency_job_id":null,"html_url":"https://github.com/xingyizhou/CenterNet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xingyizhou%2FCenterNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xingyizhou%2FCenterNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xingyizhou%2FCenterNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xingyizhou%2FCenterNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xingyizhou","download_url":"https://codeload.github.com/xingyizhou/CenterNet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243365637,"owners_count":20279215,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T21:00:47.115Z","updated_at":"2025-03-14T22:33:19.474Z","avatar_url":"https://github.com/xingyizhou.png","language":"Python","readme":"# Objects as Points\nObject detection, 3D detection, and pose estimation using center point detection:\n![](readme/fig2.png)\n\u003e [**Objects as Points**](http://arxiv.org/abs/1904.07850),            \n\u003e Xingyi Zhou, Dequan Wang, Philipp Kr\u0026auml;henb\u0026uuml;hl,        \n\u003e *arXiv technical report ([arXiv 1904.07850](http://arxiv.org/abs/1904.07850))*         \n\n\nContact: [zhouxy2017@gmail.com](mailto:zhouxy2017@gmail.com). Any questions or discussions are welcomed! \n\n## Updates\n\n - (June, 2020) We released a state-of-the-art Lidar-based 3D detection and tracking framework [CenterPoint](https://github.com/tianweiy/CenterPoint).\n - (April, 2020) We released a state-of-the-art (multi-category-/ pose-/ 3d-) tracking extension [CenterTrack](https://github.com/xingyizhou/CenterTrack).\n\n## Abstract \n\nDetection identifies objects as axis-aligned boxes in an image. Most successful object detectors enumerate a nearly exhaustive list of potential object locations and classify each. This is wasteful, inefficient, and requires additional post-processing. In this paper, we take a different approach. We model an object as a single point -- the center point of its bounding box. Our detector uses keypoint estimation to find center points and regresses to all other object properties, such as size, 3D location, orientation, and even pose. Our center point based approach, CenterNet, is end-to-end differentiable, simpler, faster, and more accurate than corresponding bounding box based detectors. CenterNet achieves the best speed-accuracy trade-off on the MS COCO dataset, with 28.1% AP at 142 FPS, 37.4% AP at 52 FPS, and 45.1% AP with multi-scale testing at 1.4 FPS. We use the same approach to estimate 3D bounding box in the KITTI benchmark and human pose on the COCO keypoint dataset. Our method performs competitively with sophisticated multi-stage methods and runs in real-time.\n\n## Highlights\n\n- **Simple:** One-sentence method summary: use keypoint detection technic to detect the bounding box center point and regress to all other object properties like bounding box size, 3d information, and pose.\n\n- **Versatile:** The same framework works for object detection, 3d bounding box estimation, and multi-person pose estimation with minor modification.\n\n- **Fast:** The whole process in a single network feedforward. No NMS post processing is needed. Our DLA-34 model runs at *52* FPS with *37.4* COCO AP.\n\n- **Strong**: Our best single model achieves *45.1*AP on COCO test-dev.\n\n- **Easy to use:** We provide user friendly testing API and webcam demos.\n\n## Main results\n\n### Object Detection on COCO validation\n\n| Backbone     |  AP / FPS | Flip AP / FPS|  Multi-scale AP / FPS |\n|--------------|-----------|--------------|-----------------------|\n|Hourglass-104 | 40.3 / 14 | 42.2 / 7.8   | 45.1 / 1.4            |\n|DLA-34        | 37.4 / 52 | 39.2 / 28    | 41.7 / 4              |\n|ResNet-101    | 34.6 / 45 | 36.2 / 25    | 39.3 / 4              |\n|ResNet-18     | 28.1 / 142| 30.0 / 71    | 33.2 / 12             |\n\n### Keypoint detection on COCO validation\n\n| Backbone     |  AP       |  FPS         |\n|--------------|-----------|--------------|\n|Hourglass-104 | 64.0      |    6.6       |\n|DLA-34        | 58.9      |    23        |\n\n### 3D bounding box detection on KITTI validation\n\n|Backbone|FPS|AP-E|AP-M|AP-H|AOS-E|AOS-M|AOS-H|BEV-E|BEV-M|BEV-H| \n|--------|---|----|----|----|-----|-----|-----|-----|-----|-----|\n|DLA-34  |32 |96.9|87.8|79.2|93.9 |84.3 |75.7 |34.0 |30.5 |26.8 |\n\n\nAll models and details are available in our [Model zoo](readme/MODEL_ZOO.md).\n\n## Installation\n\nPlease refer to [INSTALL.md](readme/INSTALL.md) for installation instructions.\n\n## Use CenterNet\n\nWe support demo for image/ image folder, video, and webcam. \n\nFirst, download the models (By default, [ctdet_coco_dla_2x](https://drive.google.com/file/d/18Q3fzzAsha_3Qid6mn4jcIFPeOGUaj1d) for detection and \n[multi_pose_dla_3x](https://drive.google.com/file/d/1mC2PAQT_RuHi_9ZMZgkt4rg7BSY2_Lkd) for human pose estimation) \nfrom the [Model zoo](readme/MODEL_ZOO.md) and put them in `CenterNet_ROOT/models/`.\n\nFor object detection on images/ video, run:\n\n~~~\npython demo.py ctdet --demo /path/to/image/or/folder/or/video --load_model ../models/ctdet_coco_dla_2x.pth\n~~~\nWe provide example images in `CenterNet_ROOT/images/` (from [Detectron](https://github.com/facebookresearch/Detectron/tree/master/demo)). If set up correctly, the output should look like\n\n\u003cp align=\"center\"\u003e \u003cimg src='readme/det1.png' align=\"center\" height=\"230px\"\u003e \u003cimg src='readme/det2.png' align=\"center\" height=\"230px\"\u003e \u003c/p\u003e\n\nFor webcam demo, run     \n\n~~~\npython demo.py ctdet --demo webcam --load_model ../models/ctdet_coco_dla_2x.pth\n~~~\n\nSimilarly, for human pose estimation, run:\n\n~~~\npython demo.py multi_pose --demo /path/to/image/or/folder/or/video/or/webcam --load_model ../models/multi_pose_dla_3x.pth\n~~~\nThe result for the example images should look like:\n\n\u003cp align=\"center\"\u003e  \u003cimg src='readme/pose1.png' align=\"center\" height=\"200px\"\u003e \u003cimg src='readme/pose2.png' align=\"center\" height=\"200px\"\u003e \u003cimg src='readme/pose3.png' align=\"center\" height=\"200px\"\u003e  \u003c/p\u003e\n\nYou can add `--debug 2` to visualize the heatmap outputs.\nYou can add `--flip_test` for flip test.\n\nTo use this CenterNet in your own project, you can \n\n~~~\nimport sys\nCENTERNET_PATH = /path/to/CenterNet/src/lib/\nsys.path.insert(0, CENTERNET_PATH)\n\nfrom detectors.detector_factory import detector_factory\nfrom opts import opts\n\nMODEL_PATH = /path/to/model\nTASK = 'ctdet' # or 'multi_pose' for human pose estimation\nopt = opts().init('{} --load_model {}'.format(TASK, MODEL_PATH).split(' '))\ndetector = detector_factory[opt.task](opt)\n\nimg = image/or/path/to/your/image/\nret = detector.run(img)['results']\n~~~\n`ret` will be a python dict: `{category_id : [[x1, y1, x2, y2, score], ...], }`\n\n## Benchmark Evaluation and Training\n\nAfter [installation](readme/INSTALL.md), follow the instructions in [DATA.md](readme/DATA.md) to setup the datasets. Then check [GETTING_STARTED.md](readme/GETTING_STARTED.md) to reproduce the results in the paper.\nWe provide scripts for all the experiments in the [experiments](experiments) folder.\n\n## Develop\n\nIf you are interested in training CenterNet in a new dataset, use CenterNet in a new task, or use a new network architecture for CenterNet, please refer to [DEVELOP.md](readme/DEVELOP.md). Also feel free to send us emails for discussions or suggestions.\n\n## Third-party resources\n\n- CenterNet + embedding learning based tracking: [FairMOT](https://github.com/ifzhang/FairMOT) from [Yifu Zhang](https://github.com/ifzhang).\n- Detectron2 based implementation: [CenterNet-better](https://github.com/FateScript/CenterNet-better) from [Feng Wang](https://github.com/FateScript).\n- Keras Implementation: [keras-centernet](https://github.com/see--/keras-centernet) from [see--](https://github.com/see--) and [keras-CenterNet](https://github.com/xuannianz/keras-CenterNet) from [xuannianz](https://github.com/xuannianz).\n- MXnet implementation: [mxnet-centernet](https://github.com/Guanghan/mxnet-centernet) from [Guanghan Ning](https://github.com/Guanghan).\n- Stronger human open estimation models: [centerpose](https://github.com/tensorboy/centerpose) from [tensorboy](https://github.com/tensorboy).\n- TensorRT extension with ONNX models: [TensorRT-CenterNet](https://github.com/CaoWGG/TensorRT-CenterNet) from [Wengang Cao](https://github.com/CaoWGG).\n- CenterNet + DeepSORT tracking implementation: [centerNet-deep-sort](https://github.com/kimyoon-young/centerNet-deep-sort) from [kimyoon-young](https://github.com/kimyoon-young).\n- Blogs on training CenterNet on custom datasets (in Chinese): [ships](https://blog.csdn.net/weixin_42634342/article/details/97756458) from [Rhett Chen](https://blog.csdn.net/weixin_42634342) and [faces](https://blog.csdn.net/weixin_41765699/article/details/100118353) from [linbior](https://me.csdn.net/weixin_41765699).\n\n\n## License\n\nCenterNet itself is released under the MIT License (refer to the LICENSE file for details).\nPortions of the code are borrowed from [human-pose-estimation.pytorch](https://github.com/Microsoft/human-pose-estimation.pytorch) (image transform, resnet), [CornerNet](https://github.com/princeton-vl/CornerNet) (hourglassnet, loss functions), [dla](https://github.com/ucbdrive/dla) (DLA network), [DCNv2](https://github.com/CharlesShang/DCNv2)(deformable convolutions), [tf-faster-rcnn](https://github.com/endernewton/tf-faster-rcnn)(Pascal VOC evaluation) and [kitti_eval](https://github.com/prclibo/kitti_eval) (KITTI dataset evaluation). Please refer to the original License of these projects (See [NOTICE](NOTICE)).\n\n## Citation\n\nIf you find this project useful for your research, please use the following BibTeX entry.\n\n    @inproceedings{zhou2019objects,\n      title={Objects as Points},\n      author={Zhou, Xingyi and Wang, Dequan and Kr{\\\"a}henb{\\\"u}hl, Philipp},\n      booktitle={arXiv preprint arXiv:1904.07850},\n      year={2019}\n    }\n","funding_links":[],"categories":["Frameworks","Python","Computer Vision"],"sub_categories":["Classification \u0026 Detection \u0026 Tracking"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxingyizhou%2FCenterNet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxingyizhou%2FCenterNet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxingyizhou%2FCenterNet/lists"}