{"id":21693528,"url":"https://github.com/kleinyuan/tf-3d-object-detection","last_synced_at":"2025-04-12T10:40:22.596Z","repository":{"id":87912034,"uuid":"134913097","full_name":"KleinYuan/tf-3d-object-detection","owner":"KleinYuan","description":"Detect object in 3D with Point Cloud and Image.","archived":false,"fork":false,"pushed_at":"2018-06-09T23:40:16.000Z","size":47,"stargazers_count":111,"open_issues_count":4,"forks_count":38,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-26T05:33:12.608Z","etag":null,"topics":["3d-object-detection","frustum-pointnet","kitti","tensorflow"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KleinYuan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-25T23:19:43.000Z","updated_at":"2024-05-22T19:05:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"154c92aa-cc31-411c-8b45-ef33423d4d5f","html_url":"https://github.com/KleinYuan/tf-3d-object-detection","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KleinYuan%2Ftf-3d-object-detection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KleinYuan%2Ftf-3d-object-detection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KleinYuan%2Ftf-3d-object-detection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KleinYuan%2Ftf-3d-object-detection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KleinYuan","download_url":"https://codeload.github.com/KleinYuan/tf-3d-object-detection/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248556541,"owners_count":21124141,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-object-detection","frustum-pointnet","kitti","tensorflow"],"created_at":"2024-11-25T18:20:39.462Z","updated_at":"2025-04-12T10:40:22.586Z","avatar_url":"https://github.com/KleinYuan.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Summary\n\n![3d](https://user-images.githubusercontent.com/8921629/41188550-0ed19016-6b74-11e8-92fb-193a8160d0e2.png)\n\n(Below is from a data in [KITTI 3D Object Detection Dataset](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d))\n\n![semi-endtoend](https://user-images.githubusercontent.com/8921629/41068890-76807090-69a0-11e8-9794-62fc394667b3.png)\n\n# Run demo\n\n#### 1. Requirements\n\n- [X] MacOS or Ubuntu\n\n- [X] Tensorflow\n\n- [X] Mayavi (visualization Only)\n\n- [X] OpenCV\n\n- [ ] Anaconda preferred (optional)\n\n### 2. Clone this repo\n\n```\ngit clone https://github.com/KleinYuan/tf-3d-object-detection.git\n```\n\n### 2. Install Dependencies\n\n```\n# Simply run this in this project root folder\ncd tf-3d-object-detection\npip install -r requirements.txt\n```\n\nIf you meet error install say `opencv`, do `conda install opencv` if you use Anaconda. Otherwise, dude, build from source and let's call it a day.\n\n### 3. Pick a 2D Object Detection Model\n\nIn here we support 5 different 2D Detection models:\n\n| Model name  | Speed | COCO mAP | Outputs |\n| ------------ | :--------------: | :--------------: | :-------------: |\n| [ssd_mobilenet_v1_coco](http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_11_06_2017.tar.gz) | fast | 21 | Boxes |\n| [ssd_inception_v2_coco](http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_11_06_2017.tar.gz) | fast | 24 | Boxes |\n| [rfcn_resnet101_coco](http://download.tensorflow.org/models/object_detection/rfcn_resnet101_coco_11_06_2017.tar.gz)  | medium | 30 | Boxes |\n| [faster_rcnn_resnet101_coco](http://download.tensorflow.org/models/object_detection/faster_rcnn_resnet101_coco_11_06_2017.tar.gz) | medium | 32 | Boxes |\n| [faster_rcnn_inception_resnet_v2_atrous_coco](http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_resnet_v2_atrous_coco_11_06_2017.tar.gz) | slow | 37 | Boxes |\n\nPick one of those that makes you feel good, and find it in the list -- [`_DETECTOR_2D_OPTIONS` in `configs/configs`](https://github.com/KleinYuan/tf-3d-object-detection/blob/master/configs/configs.py#L17),\nthen replace it with the value of [`_DETECTOR_2D_MODEL_NAME`](https://github.com/KleinYuan/tf-3d-object-detection/blob/master/configs/configs.py#L16).\n\nAnd by default, I use [`ssd_mobilenet_v1_coco_11_06_2017`](https://github.com/KleinYuan/tf-3d-object-detection/blob/master/configs/configs.py#L16) due to it's fast.\n\n### 4. Download Test Data\n\nDue to the license of KITTI is waaaaaaaaaaay to long to read, I will just tell ya how to do it instead of running a risk to attach here with some data from KITTI, which\nwhen I downloaded it I clicked some button to have agreed on something that's TLTR.\n\n```\n# Step1 Go to http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d\n# Step2 Do \"Download left color images of object data set (12 GB)\"\n# Step3 Do \"Download Velodyne point clouds, if you want to use laser information (29 GB)\"\n# Step4 Do \"Download camera calibration matrices of object data set (16 MB)\"\n# Step5 Unzip all those three zip files and you will find ~7000ish training datasets, each pair include velodyne, image and calibration\n# Step6 Pick one of them, copy and paste it under example_data folder, and name the image to 1.png, and velodyne file to 1.bin\n# Step7 Open calibration file, find corresponding item and replace it with CALIB_PARAM in configs/configs.py, by default, it's from training/000000.txt\n# Step8 Really sorry to let you go thru last 7 Steps and I think I may come up with a better idea to do it with one button\n```\n\n### 5. Download Pretrained Model\n\nAs you may see, this project combined 2 Deep Neural Networks together. Therefore, yes you need to download two pre-trained model.\n\n| 2D Object Detector Model  | 3D Object Detector Model |\n| ------------ | :--------------: |\n| [Download Link](https://github.com/KleinYuan/tf-object-detection/blob/master/README.md#introduction)| [Download v1 and v2 is not supported yet](https://shapenet.cs.stanford.edu/media/frustum_pointnets_snapshots.zip) (originally from [here](https://github.com/Dark-Rinnegan/frustum-pointnets/tree/app#training-frustum-pointnets))|\n\nThen, unzip them and put them under [`pretrained`](https://github.com/KleinYuan/tf-3d-object-detection/tree/master/pretrained) folder. Also, renamed the `checkpoint.txt` file to `checkpoint` even though it's useless and you cannot freeze it :unhappy: .\n\n\nThe folder will look like this:\n\n```\n--tf-3d-object-detection\n  |-- pretrained\n      |--log_v1\n          |-- checkpoint (originally named checkpoint.txt)\n          |-- log_train.txt\n          |-- model.ckpt.data-00000-of-00001\n          |-- model.ckpt\n          |-- model.ckpt.meta\n      |-- ssd_mobilenet_v1_coco_11_06_2017 (or other names if you decide to use different ones)\n          |-- frozen_inference_graph.pb\n          |-- graph.pbtxt\n          |-- model.ckpt-0.data-00000-of-00001\n          |-- model.ckpt-0.index\n          |-- model.ckpt-0.meta\n\n```\n\nYou may realize this [fact](https://github.com/Dark-Rinnegan/frustum-pointnets/tree/app/app#intro) thus 3D object detection model is not really frozenable one.\n\n(Hopefully they can disclose the original tensorflow ops for v1 so that we can remove both `tf.py_func` and freeze the model)\n\n### 6. Run Demo\n\n```\n# If you use Pycharm, just click the green run button\n# If not, navigate to root folder of this repo and run:\npython apps/demo.py\n\n# If it complains, yo, I cannot find some modules, yo, do:\nexport PYTHONPATH='.'\npython apps/demo.py\n\n# And if you still have the issue, man, you must really mess up with your python env.\n# I don't wanan help you on that in this readme and don't create an issue for that as well.\n# You shall either try using anaconda or find a python knower to help you with it\n# Or, just do STACKOVERFLOW like other pals do\n\n```\n\nThen you should be able to see 3 Windows pop up in order, and don't forget to `Press any key to continue` as the terminal mention.\n\n\n# References\n\n- [X] Project Template: [AIInAi/tensorflow-project-template](https://github.com/AIInAi/tensorflow-project-template)\n\n- [X] FPNet Code: [Dark-Rinnegan/frustum-pointnets](https://github.com/Dark-Rinnegan/frustum-pointnets/tree/app/app)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkleinyuan%2Ftf-3d-object-detection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkleinyuan%2Ftf-3d-object-detection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkleinyuan%2Ftf-3d-object-detection/lists"}