{"id":13443011,"url":"https://github.com/skhadem/3D-BoundingBox","last_synced_at":"2025-03-20T15:31:55.260Z","repository":{"id":45318721,"uuid":"170003722","full_name":"skhadem/3D-BoundingBox","owner":"skhadem","description":"PyTorch implementation for 3D Bounding Box Estimation Using Deep Learning and Geometry","archived":false,"fork":false,"pushed_at":"2024-01-30T21:39:40.000Z","size":16757,"stargazers_count":443,"open_issues_count":26,"forks_count":96,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-10-28T06:00:25.299Z","etag":null,"topics":["3d","computer-vision","localization","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/skhadem.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-10T17:15:32.000Z","updated_at":"2024-10-22T03:01:58.000Z","dependencies_parsed_at":"2024-10-28T04:01:31.862Z","dependency_job_id":null,"html_url":"https://github.com/skhadem/3D-BoundingBox","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skhadem%2F3D-BoundingBox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skhadem%2F3D-BoundingBox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skhadem%2F3D-BoundingBox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skhadem%2F3D-BoundingBox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/skhadem","download_url":"https://codeload.github.com/skhadem/3D-BoundingBox/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244640129,"owners_count":20485990,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","computer-vision","localization","pytorch"],"created_at":"2024-07-31T03:01:54.762Z","updated_at":"2025-03-20T15:31:53.425Z","avatar_url":"https://github.com/skhadem.png","language":"Python","funding_links":[],"categories":["Python","3. Perception"],"sub_categories":["3.1.1 Vision based"],"readme":"# 3D Bounding Box Estimation Using Deep Learning and Geometry\nIf interested, join the slack workspace where the paper is discussed, issues are worked through, and more! Click this [link](https://join.slack.com/t/3dboundingbox-oun9186/shared_invite/enQtNDk4Njg2NzYyNzY5LWVlZWRlMjNhZmZlYjVmNGY3NWVlNDA4MmY2ZWQ3ZmUyY2Q4OWIxMmY4NzU4YmViM2ViZWI5YjgxOTIyOTI4ZjI) to join.\n\n## Introduction\nPyTorch implementation for this [paper](https://arxiv.org/abs/1612.00496).\n\n![example-image](http://soroushkhadem.com/img/2d-top-3d-bottom1.png)\n\nAt the moment, it takes approximately 0.4s per frame, depending on the number of objects\ndetected. An improvement will be speed upgrades soon. Here is the current fastest possible:\n![example-video](eval/example/3d-bbox-vid.gif)\n\n## Requirements\n- PyTorch\n- Cuda\n- OpenCV \u003e= 3.4.3\n\n## Usage\nIn order to download the weights:\n```\ncd weights/\n./get_weights.sh\n```\nThis will download pre-trained weights for the 3D BoundingBox net and also YOLOv3 weights from the\nofficial yolo [source](https://pjreddie.com/darknet/yolo/).\n\n\u003eIf script is not working: [pre trained weights](https://drive.google.com/open?id=1yEiquJg9inIFgR3F-N5Z3DbFnXJ0aXmA) and \n[YOLO weights](https://pjreddie.com/media/files/yolov3.weights)\n\nTo see all the options:\n```\npython Run.py --help\n```\n\nRun through all images in default directory (eval/image_2/), optionally with the 2D\nbounding boxes also drawn. Press SPACE to proceed to next image, and any other key to exit.\n```\npython Run.py [--show-yolo]\n```\n\u003eNote: See [training](#training) for where to download the data from\n\nThere is also a script provided to download the default video from Kitti in ./eval/video. Or,\ndownload any Kitti video and corresponding calibration and use `--image-dir` and `--cal-dir` to\nspecify where to get the frames from.\n```\npython Run.py --video [--hide-debug]\n```\n\n## Training\nFirst, the data must be downloaded from [Kitti](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=2d).\nDownload the left color images, the training labels, and the camera calibration matrices. Total is ~13GB.\nUnzip the downloads into the Kitti/ directory.\n\n```\npython Train.py\n```\nBy default, the model is saved every 10 epochs in weights/.\nThe loss is printed every 10 batches. The loss should not converge to 0! The loss function for\nthe orientation is driven to -1, so a negative loss is expected. The hyper-parameters to tune\nare alpha and w (see paper). I obtained good results after just 10 epochs, but the training\nscript will run until 100.\n\n## How it works\nThe PyTorch neural net takes in images of size 224x224 and predicts the orientation and\nrelative dimension of that object to the class average. Thus, another neural net must give\nthe 2D bounding box and object class. I chose to use YOLOv3 through OpenCV.\nUsing the orientation, dimension, and 2D bounding box, the 3D location is calculated, and then\nback projected onto the image.\n\nThere are 2 key assumptions made:\n1. The 2D bounding box fits very tightly around the object\n2. The object has ~0 pitch and ~0 roll (valid for cars on the road)\n\n## Future Goals\n- Train custom YOLO net on the Kitti dataset\n- Some type of Pose visualization (ROS?)\n\n## Credit\n1. I originally started from a fork of this [repo](https://github.com/fuenwang/3D-BoundingBox), and some of the original code still exists in the training script.\n2. 2D-3D geometric [conversion](https://ywpkwon.github.io/pdf/bbox3d-study.pdf).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskhadem%2F3D-BoundingBox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fskhadem%2F3D-BoundingBox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskhadem%2F3D-BoundingBox/lists"}