{"id":13439155,"url":"https://github.com/lijx10/DeepI2P","last_synced_at":"2025-03-20T07:32:34.319Z","repository":{"id":43665556,"uuid":"351784259","full_name":"lijx10/DeepI2P","owner":"lijx10","description":"DeepI2P: Image-to-Point Cloud Registration via Deep Classification. CVPR 2021","archived":false,"fork":false,"pushed_at":"2023-06-01T02:43:10.000Z","size":13324,"stargazers_count":206,"open_issues_count":15,"forks_count":36,"subscribers_count":8,"default_branch":"main","last_synced_at":"2024-08-01T03:28:43.710Z","etag":null,"topics":["3d-vision","cnn","computer-vision","deep-learning","image-processing","localization","neural-network","point-cloud","registration"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lijx10.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-03-26T13:03:18.000Z","updated_at":"2024-07-26T05:12:20.000Z","dependencies_parsed_at":"2024-01-16T02:01:35.947Z","dependency_job_id":"80d61852-ad57-4d35-a8ce-a58c1cb4fd70","html_url":"https://github.com/lijx10/DeepI2P","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lijx10%2FDeepI2P","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lijx10%2FDeepI2P/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lijx10%2FDeepI2P/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lijx10%2FDeepI2P/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lijx10","download_url":"https://codeload.github.com/lijx10/DeepI2P/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221739666,"owners_count":16872768,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-vision","cnn","computer-vision","deep-learning","image-processing","localization","neural-network","point-cloud","registration"],"created_at":"2024-07-31T03:01:11.583Z","updated_at":"2024-10-27T22:30:26.404Z","avatar_url":"https://github.com/lijx10.png","language":"C++","funding_links":[],"categories":["C++"],"sub_categories":[],"readme":"#DeepI2P: Image-to-Point Cloud Registration via Deep Classification\n## Summary\n### [Video](resources/deepi2p-cvpr2021-compressed.mp4)\nPyTorch implementation for our CVPR 2021 paper DeepI2P. DeepI2P solves the problem of cross modality registration, i.e, solve the relative rotation `R` and translation `t` between the camera and the lidar.\n\nDeepI2P: Image-to-Point Cloud Registration via Deep Classification\u003cbr\u003e\n[Jiaxin Li](https://www.jiaxinli.me/) \u003csup\u003e1\u003c/sup\u003e,\n[Gim Hee Lee](https://www.comp.nus.edu.sg/~leegh) \u003csup\u003e2\u003c/sup\u003e \u003cbr\u003e\n\u003csup\u003e1\u003c/sup\u003eByteDance, \u003csup\u003e2\u003c/sup\u003eNational University of Singapore  \n\n### Method\nThe intuition is to perform the ``Inverse Camera Projection``, as shown in the images below.\n![overview_1](resources/overview_1.png)\n![overview_2](resources/overview_2.png)\n\n## Repo Structure\n- `data`: Generate and process datasets\n- `evaluation`: Registration codes, include Inverse Camera Projection, ICP, PnP\n  - `frustum_reg`: C++ codes of the Inverse Camera Projection, using Gauss-Newton Optimization. Installation method is shown below. It requires the [Ceres Solver](http://ceres-solver.org/). \n  ```\n  python evaluation/frustum_reg/setup.py install\n  ```\n  - `icp`: codes for ICP (Iterative Closest Point)\n  - `registration_lsq.py`: Python code for Inverse Camera Projection, which utilizes the per-point coarse classification prediction, and the `frustum_reg` solver.\n  - `registration_pnp.py`: Python code for PnP solver utilizing the per-point fine classification prediction.\n- `kitti`: Training codes for KITTI\n- `nuscenes`: Training codes for nuscenes\n- `oxford`: Training codes for Oxford Robotcar dataset\n- `models`: Networks and layers\n  - 'index_max_ext': This is a custom operation from [SO-Net](https://github.com/lijx10/SO-Net), which is the backbone of our network. Installation:\n  ```\n  python models/index_max_ext/setup.py install\n  ```\n  - `networks_img.py`: Network to process images. It is a resnet-like structure.\n  - `networks_pc.py`: Network to process point clouds, it is from [SO-Net](https://github.com/lijx10/SO-Net)\n  - `network_united.py`: Network to fuse information between point clouds and images.\n\n## Dataset and Models\n- [Oxford train/test split and evaluation dataset](https://drive.google.com/drive/folders/1MVNDGCRXVjC-hxtW_mg4abMXmtpNVzLR?usp=sharing)\n- [Oxford coarse classification model](https://drive.google.com/drive/folders/1rycR7xtptDA9gqh-p4n5AsZtu0d3iKsC?usp=sharing)\n- [Oxford coarse \\\u0026 fine classification model](https://drive.google.com/drive/folders/1lzbI1PjmqmmdiVYGgkSB1yo7-7vnW9f4?usp=sharing)\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flijx10%2FDeepI2P","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flijx10%2FDeepI2P","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flijx10%2FDeepI2P/lists"}