{"id":15033482,"url":"https://github.com/maudzung/complex-yolov4-pytorch","last_synced_at":"2025-04-08T02:42:50.904Z","repository":{"id":37642854,"uuid":"277005144","full_name":"maudzung/Complex-YOLOv4-Pytorch","owner":"maudzung","description":"The PyTorch Implementation based on YOLOv4 of the paper: \"Complex-YOLO: Real-time 3D Object Detection on Point Clouds\"","archived":false,"fork":false,"pushed_at":"2024-08-30T23:53:42.000Z","size":6813,"stargazers_count":1276,"open_issues_count":57,"forks_count":258,"subscribers_count":26,"default_branch":"master","last_synced_at":"2025-04-08T02:42:44.104Z","etag":null,"topics":["3d-object-detection","complex-yolo","data-parallel-computing","giou","lidar","lidar-point-cloud","mish","mosaic","multiprocessing","object-detection","real-time","rotated-boxes","rotated-boxes-iou","yolov4"],"latest_commit_sha":null,"homepage":"https://arxiv.org/pdf/1803.06199.pdf","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maudzung.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-03T23:39:23.000Z","updated_at":"2025-04-03T08:27:48.000Z","dependencies_parsed_at":"2023-01-31T09:46:02.540Z","dependency_job_id":"0fc99e5d-3160-4686-9975-d85b70e56c85","html_url":"https://github.com/maudzung/Complex-YOLOv4-Pytorch","commit_stats":{"total_commits":123,"total_committers":4,"mean_commits":30.75,"dds":"0.024390243902439046","last_synced_commit":"2f32996fc4d8942a156545596f76bbc267d5cd8e"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maudzung%2FComplex-YOLOv4-Pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maudzung%2FComplex-YOLOv4-Pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maudzung%2FComplex-YOLOv4-Pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maudzung%2FComplex-YOLOv4-Pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maudzung","download_url":"https://codeload.github.com/maudzung/Complex-YOLOv4-Pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247767232,"owners_count":20992538,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-object-detection","complex-yolo","data-parallel-computing","giou","lidar","lidar-point-cloud","mish","mosaic","multiprocessing","object-detection","real-time","rotated-boxes","rotated-boxes-iou","yolov4"],"created_at":"2024-09-24T20:21:25.089Z","updated_at":"2025-04-08T02:42:50.879Z","avatar_url":"https://github.com/maudzung.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Complex YOLOv4\n\n[![python-image]][python-url]\n[![pytorch-image]][pytorch-url]\n\nThe PyTorch Implementation based on YOLOv4 of the paper: [Complex-YOLO: Real-time 3D Object Detection on Point Clouds](https://arxiv.org/pdf/1803.06199.pdf)\n\n---\n\n## Features\n- [x] Realtime 3D object detection based on YOLOv4\n- [x] Support [distributed data parallel training](https://github.com/pytorch/examples/tree/master/distributed/ddp)\n- [x] Tensorboard\n- [x] Mosaic/Cutout augmentation for training\n- [x] Use [GIoU](https://arxiv.org/pdf/1902.09630v2.pdf) loss of rotated boxes for optimization.\n\n- **Update 2020.08.26**: [Super Fast and Accurate 3D Object Detection based on 3D LiDAR Point Clouds](https://github.com/maudzung/Super-Fast-Accurate-3D-Object-Detection)\n    - Faster training, faster inference\n    \n    - An Anchor-free approach\n    \n    - No need for Non-Max-Suppression\n    \n    - Demonstration (on a GTX 1080Ti)\n\n[![demo](http://img.youtube.com/vi/FI8mJIXkgX4/0.jpg)](http://www.youtube.com/watch?v=FI8mJIXkgX4)\n\n\n**[Youtube link](https://youtu.be/FI8mJIXkgX4)**\n\n## 2. Getting Started\n### 2.1. Requirement\n\n```shell script\npip install -U -r requirements.txt\n```\n\nFor [`mayavi`](https://docs.enthought.com/mayavi/mayavi/installation.html) and [`shapely`](https://shapely.readthedocs.io/en/latest/project.html#installing-shapely) \nlibraries, please refer to the installation instructions from their official websites.\n\n\n### 2.2. Data Preparation\nDownload the 3D KITTI detection dataset from [here](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d).\n\nThe downloaded data includes:\n\n- Velodyne point clouds _**(29 GB)**_: input data to the Complex-YOLO model\n- Training labels of object data set _**(5 MB)**_: input label to the Complex-YOLO model\n- Camera calibration matrices of object data set _**(16 MB)**_: for visualization of predictions\n- Left color images of object data set _**(12 GB)**_: for visualization of predictions\n\nPlease make sure that you construct the source code \u0026 dataset directories structure as below.\n\nFor 3D point cloud preprocessing, please refer to the previous works:\n- [VoxelNet-Pytorch](https://github.com/skyhehe123/VoxelNet-pytorch)\n- [Complex-YOLOv2](https://github.com/AI-liu/Complex-YOLO)\n- [Complex-YOLOv3](https://github.com/ghimiredhikura/Complex-YOLOv3)\n\n\n### 2.3. Complex-YOLO architecture\n\n![architecture](./docs/complex_yolo_architecture.PNG)\n\nThis work has been based on the paper [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://arxiv.org/abs/2004.10934).\n\nPlease refer to several implementations of YOLOv4 using PyTorch DL framework:\n- [Tianxiaomo/pytorch-YOLOv4](https://github.com/Tianxiaomo/pytorch-YOLOv4)\n- [Ultralytics/yolov3_and_v4](https://github.com/ultralytics/yolov3)\n- [WongKinYiu/PyTorch_YOLOv4](https://github.com/WongKinYiu/PyTorch_YOLOv4)\n- [VCasecnikovs/Yet-Another-YOLOv4-Pytorch](https://github.com/VCasecnikovs/Yet-Another-YOLOv4-Pytorch)\n\n### 2.4. How to run\n\n#### 2.4.1. Visualize the dataset (both BEV images from LiDAR and camera images)\n\n```shell script\ncd src/data_process\n```\n\n- To visualize BEV maps and camera images (with 3D boxes), let's execute _**(the `output-width` param can be changed to \nshow the images in a bigger/smaller window)**_:\n\n```shell script\npython kitti_dataloader.py --output-width 608\n```\n\n- To visualize mosaics that are composed from 4 BEV maps (Using during training only), let's execute:\n\n```shell script\npython kitti_dataloader.py --show-train-data --mosaic --output-width 608 \n```\n\nBy default, there is _**no padding**_ for the output mosaics, the feature could be activated by executing:\n\n```shell script\npython kitti_dataloader.py --show-train-data --mosaic --random-padding --output-width 608 \n```\n\n- To visualize cutout augmentation, let's execute:\n\n```shell script\npython kitti_dataloader.py --show-train-data --cutout_prob 1. --cutout_nholes 1 --cutout_fill_value 1. --cutout_ratio 0.3 --output-width 608\n```\n\n#### 2.4.2. Inference\n\nDownload the trained model from [**_here_**](https://drive.google.com/drive/folders/1RHD9PBvk-9SjbKwoi_Q1kl9-UGFo2Pth?usp=sharing), \nthen put it to `${ROOT}/checkpoints/` and execute:\n\n```shell script\npython test.py --gpu_idx 0 --pretrained_path ../checkpoints/complex_yolov4/complex_yolov4_mse_loss.pth --cfgfile ./config/cfg/complex_yolov4.cfg --show_image\n```\n\n#### 2.4.3. Evaluation\n\n```shell script\npython evaluate.py --gpu_idx 0 --pretrained_path \u003cPATH\u003e --cfgfile \u003cCFG\u003e --img_size \u003cSIZE\u003e --conf-thresh \u003cTHRESH\u003e --nms-thresh \u003cTHRESH\u003e --iou-thresh \u003cTHRESH\u003e\n```\n(The `conf-thresh`, `nms-thresh`, and `iou-thresh` params can be adjusted. By default, these params have been set to _**0.5**_)\n\n#### 2.4.4. Training\n\n##### 2.4.4.1. Single machine, single gpu\n\n```shell script\npython train.py --gpu_idx 0 --batch_size \u003cN\u003e --num_workers \u003cN\u003e...\n```\n\n##### 2.4.4.2. Multi-processing Distributed Data Parallel Training\nWe should always use the `nccl` backend for multi-processing distributed training since it currently provides the best \ndistributed training performance.\n\n- **Single machine (node), multiple GPUs**\n\n```shell script\npython train.py --dist-url 'tcp://127.0.0.1:29500' --dist-backend 'nccl' --multiprocessing-distributed --world-size 1 --rank 0\n```\n\n- **Two machines (two nodes), multiple GPUs**\n\n_**First machine**_\n\n```shell script\npython train.py --dist-url 'tcp://IP_OF_NODE1:FREEPORT' --dist-backend 'nccl' --multiprocessing-distributed --world-size 2 --rank 0\n```\n_**Second machine**_\n\n```shell script\npython train.py --dist-url 'tcp://IP_OF_NODE2:FREEPORT' --dist-backend 'nccl' --multiprocessing-distributed --world-size 2 --rank 1\n```\n\nTo reproduce the results, you can run the bash shell script\n\n```bash\n./train.sh\n```\n\n#### Tensorboard\n\n- To track the training progress, go to the `logs/` folder and \n\n```shell script\ncd logs/\u003csaved_fn\u003e/tensorboard/\ntensorboard --logdir=./\n```\n\n- Then go to [http://localhost:6006/](http://localhost:6006/):\n\n\n### 2.5. List of usage for Bag of Freebies (BoF) \u0026 Bag of Specials (BoS) in this implementation\n\n\n|   |Backbone   | Detector   |\n|---|---|---|\n|**BoF**   |[x] Dropblock \u003cbr\u003e [x] Random rescale, rotation (global) \u003cbr\u003e [x] Mosaic/Cutout augmentation|[x] Cross mini-Batch Normalization \u003cbr\u003e[x] Dropblock \u003cbr\u003e [x] Random training shapes \u003cbr\u003e   |\n|**BoS**   |[x] Mish activation \u003cbr\u003e [x] Cross-stage partial connections (CSP) \u003cbr\u003e [x] Multi-input weighted residual connections (MiWRC)   |[x] Mish activation \u003cbr\u003e [x] SPP-block \u003cbr\u003e [x] SAM-block \u003cbr\u003e [x] PAN path-aggregation block \u003cbr\u003e [x] GIoU loss \u003cbr\u003e [ ] CIoU loss |\n\n\n## Contact\n\nIf you think this work is useful, please give me a star! \u003cbr\u003e\nIf you find any errors or have any suggestions, please contact me (**Email:** `nguyenmaudung93.kstn@gmail.com`). \u003cbr\u003e\nThank you!\n\n\n## Citation\n\n```bash\n@article{Complex-YOLO,\n  author = {Martin Simon, Stefan Milz, Karl Amende, Horst-Michael Gross},\n  title = {Complex-YOLO: Real-time 3D Object Detection on Point Clouds},\n  year = {2018},\n  journal = {arXiv},\n}\n\n@article{YOLOv4,\n  author = {Alexey Bochkovskiy, Chien-Yao Wang, Hong-Yuan Mark Liao},\n  title = {YOLOv4: Optimal Speed and Accuracy of Object Detection},\n  year = {2020},\n  journal = {arXiv},\n}\n```\n\n\n## Folder structure\n\n```\n${ROOT}\n└── checkpoints/    \n    ├── complex_yolov3/\n    └── complex_yolov4/\n└── dataset/    \n    └── kitti/\n        ├──ImageSets/\n        │   ├── train.txt\n        │   └── val.txt\n        ├── training/\n        │   ├── image_2/ \u003c-- for visualization\n        │   ├── calib/\n        │   ├── label_2/\n        │   └── velodyne/\n        └── testing/  \n        │   ├── image_2/ \u003c-- for visualization\n        │   ├── calib/\n        │   └── velodyne/ \n        └── classes_names.txt\n└── src/\n    ├── config/\n    ├── cfg/\n        │   ├── complex_yolov3.cfg\n        │   ├── complex_yolov3_tiny.cfg\n        │   ├── complex_yolov4.cfg\n        │   ├── complex_yolov4_tiny.cfg\n    │   ├── train_config.py\n    │   └── kitti_config.py\n    ├── data_process/\n    │   ├── kitti_bev_utils.py\n    │   ├── kitti_dataloader.py\n    │   ├── kitti_dataset.py\n    │   ├── kitti_data_utils.py\n    │   ├── train_val_split.py\n    │   └── transformation.py\n    ├── models/\n    │   ├── darknet2pytorch.py\n    │   ├── darknet_utils.py\n    │   ├── model_utils.py\n    │   ├── yolo_layer.py\n    └── utils/\n    │   ├── evaluation_utils.py\n    │   ├── iou_utils.py\n    │   ├── logger.py\n    │   ├── misc.py\n    │   ├── torch_utils.py\n    │   ├── train_utils.py\n    │   └── visualization_utils.py\n    ├── evaluate.py\n    ├── test.py\n    ├── test.sh\n    ├── train.py\n    └── train.sh\n├── README.md \n└── requirements.txt\n```\n\n## Usage\n\n```\nusage: train.py [-h] [--seed SEED] [--saved_fn FN] [--working-dir PATH]\n                [-a ARCH] [--cfgfile PATH] [--pretrained_path PATH]\n                [--img_size IMG_SIZE] [--hflip_prob HFLIP_PROB]\n                [--cutout_prob CUTOUT_PROB] [--cutout_nholes CUTOUT_NHOLES]\n                [--cutout_ratio CUTOUT_RATIO]\n                [--cutout_fill_value CUTOUT_FILL_VALUE]\n                [--multiscale_training] [--mosaic] [--random-padding]\n                [--no-val] [--num_samples NUM_SAMPLES]\n                [--num_workers NUM_WORKERS] [--batch_size BATCH_SIZE]\n                [--print_freq N] [--tensorboard_freq N] [--checkpoint_freq N]\n                [--start_epoch N] [--num_epochs N] [--lr_type LR_TYPE]\n                [--lr LR] [--minimum_lr MIN_LR] [--momentum M] [-wd WD]\n                [--optimizer_type OPTIMIZER] [--burn_in N]\n                [--steps [STEPS [STEPS ...]]] [--world-size N] [--rank N]\n                [--dist-url DIST_URL] [--dist-backend DIST_BACKEND]\n                [--gpu_idx GPU_IDX] [--no_cuda]\n                [--multiprocessing-distributed] [--evaluate]\n                [--resume_path PATH] [--conf-thresh CONF_THRESH]\n                [--nms-thresh NMS_THRESH] [--iou-thresh IOU_THRESH]\n\nThe Implementation of Complex YOLOv4\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --seed SEED           re-produce the results with seed random\n  --saved_fn FN         The name using for saving logs, models,...\n  --working-dir PATH    The ROOT working directory\n  -a ARCH, --arch ARCH  The name of the model architecture\n  --cfgfile PATH        The path for cfgfile (only for darknet)\n  --pretrained_path PATH\n                        the path of the pretrained checkpoint\n  --img_size IMG_SIZE   the size of input image\n  --hflip_prob HFLIP_PROB\n                        The probability of horizontal flip\n  --cutout_prob CUTOUT_PROB\n                        The probability of cutout augmentation\n  --cutout_nholes CUTOUT_NHOLES\n                        The number of cutout area\n  --cutout_ratio CUTOUT_RATIO\n                        The max ratio of the cutout area\n  --cutout_fill_value CUTOUT_FILL_VALUE\n                        The fill value in the cut out area, default 0. (black)\n  --multiscale_training\n                        If true, use scaling data for training\n  --mosaic              If true, compose training samples as mosaics\n  --random-padding      If true, random padding if using mosaic augmentation\n  --no-val              If true, dont evaluate the model on the val set\n  --num_samples NUM_SAMPLES\n                        Take a subset of the dataset to run and debug\n  --num_workers NUM_WORKERS\n                        Number of threads for loading data\n  --batch_size BATCH_SIZE\n                        mini-batch size (default: 4), this is the totalbatch\n                        size of all GPUs on the current node when usingData\n                        Parallel or Distributed Data Parallel\n  --print_freq N        print frequency (default: 50)\n  --tensorboard_freq N  frequency of saving tensorboard (default: 20)\n  --checkpoint_freq N   frequency of saving checkpoints (default: 2)\n  --start_epoch N       the starting epoch\n  --num_epochs N        number of total epochs to run\n  --lr_type LR_TYPE     the type of learning rate scheduler (cosin or\n                        multi_step)\n  --lr LR               initial learning rate\n  --minimum_lr MIN_LR   minimum learning rate during training\n  --momentum M          momentum\n  -wd WD, --weight_decay WD\n                        weight decay (default: 1e-6)\n  --optimizer_type OPTIMIZER\n                        the type of optimizer, it can be sgd or adam\n  --burn_in N           number of burn in step\n  --steps [STEPS [STEPS ...]]\n                        number of burn in step\n  --world-size N        number of nodes for distributed training\n  --rank N              node rank for distributed training\n  --dist-url DIST_URL   url used to set up distributed training\n  --dist-backend DIST_BACKEND\n                        distributed backend\n  --gpu_idx GPU_IDX     GPU index to use.\n  --no_cuda             If true, cuda is not used.\n  --multiprocessing-distributed\n                        Use multi-processing distributed training to launch N\n                        processes per node, which has N GPUs. This is the\n                        fastest way to use PyTorch for either single node or\n                        multi node data parallel training\n  --evaluate            only evaluate the model, not training\n  --resume_path PATH    the path of the resumed checkpoint\n  --conf-thresh CONF_THRESH\n                        for evaluation - the threshold for class conf\n  --nms-thresh NMS_THRESH\n                        for evaluation - the threshold for nms\n  --iou-thresh IOU_THRESH\n                        for evaluation - the threshold for IoU\n```\n\n[python-image]: https://img.shields.io/badge/Python-3.6-ff69b4.svg\n[python-url]: https://www.python.org/\n[pytorch-image]: https://img.shields.io/badge/PyTorch-1.5-2BAF2B.svg\n[pytorch-url]: https://pytorch.org/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaudzung%2Fcomplex-yolov4-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaudzung%2Fcomplex-yolov4-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaudzung%2Fcomplex-yolov4-pytorch/lists"}