{"id":20041728,"url":"https://github.com/ajithvcoder/planercnn_customimages","last_synced_at":"2026-03-07T00:30:51.802Z","repository":{"id":111987352,"uuid":"315013875","full_name":"ajithvcoder/planercnn_customimages","owner":"ajithvcoder","description":"Contains codes for training placercnn for custom images","archived":false,"fork":false,"pushed_at":"2020-12-13T08:12:51.000Z","size":9624,"stargazers_count":5,"open_issues_count":1,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-05T08:41:11.974Z","etag":null,"topics":["3dreconstruction","planercnn","planes","pytorch","segmentations"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ajithvcoder.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-11-22T10:45:55.000Z","updated_at":"2025-03-24T07:24:16.000Z","dependencies_parsed_at":null,"dependency_job_id":"d30e6871-c879-4f1f-b476-13e9ca1593b4","html_url":"https://github.com/ajithvcoder/planercnn_customimages","commit_stats":null,"previous_names":["ajithvcoder/planercnn_customimages"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ajithvcoder/planercnn_customimages","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2Fplanercnn_customimages","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2Fplanercnn_customimages/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2Fplanercnn_customimages/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2Fplanercnn_customimages/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ajithvcoder","download_url":"https://codeload.github.com/ajithvcoder/planercnn_customimages/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2Fplanercnn_customimages/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30204154,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T19:07:06.838Z","status":"ssl_error","status_checked_at":"2026-03-06T18:57:34.882Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3dreconstruction","planercnn","planes","pytorch","segmentations"],"created_at":"2024-11-13T10:47:38.275Z","updated_at":"2026-03-07T00:30:51.783Z","avatar_url":"https://github.com/ajithvcoder.png","language":"Python","readme":"[![License CC BY-NC-SA 4.0](https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-blue.svg)](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode)\n![Python 3.7](https://img.shields.io/badge/python-3.7-green.svg)\n\n# PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image - Extended\n\n\n## Introduction\n\nThis paper proposes a deep neural architecture, PlaneR-CNN, that detects arbitrary number of planes, and reconstructs piecewise planar surfaces from a single RGB image. The code is implemented using PyTorch.\n\n\n\n## Getting Started \nClone repository: \n```\ngit clone https://github.com/NVlabs/planercnn.git\n```\n\nPlease use Python 3. Create an [Anaconda](https://www.anaconda.com/distribution/) environment and install the dependencies:\n```\nconda create --name planercnn\nconda activate planercnn\nconda install -y pytorch=0.4.1\nconda install pip\npip install -r requirements.txt\n```\nEquivalently, you can use Python virtual environment to manage the dependencies:\n```\npip install virtualenv\npython -m virtualenv planercnn\nsource planercnn/bin/activate\npip install -r requirements.txt\n```\n\nNote: If you are doing in colab then go with cuda8.0 , torch 0.4.0 , gcc 5+ for building and upgrade to torch 0.4.1 for evaluate/training.\n\n\nPlease note that, the Mask R-CNN backbone does not support cuda10.0 and gcc versions higher than 7. If you have troubles compiling these two libraries, try to downgrade PyTorch to 0.4.0 before compilation and upgrade back to 0.4.1 after compilation. You might also want to find more information on their original [repository](https://github.com/multimodallearning/pytorch-mask-rcnn).\n\n## Models - verified\nModels are saved under *checkpoint/*. You can download our trained model from [here](https://www.dropbox.com/s/yjcg6s57n581sk0/checkpoint.zip?dl=0), and put it under *checkpoint/* if you want to fine-tune it or run inferences.\n\nBackup gdrive link - [here](https://drive.google.com/file/d/1o2wZG0swF-HImZbQGPC7cHONkCThFVQZ/view?usp=sharing)\n\n## Plane representation\nIn this project, plane parameters are of absolute scale (in terms of meters). Each plane has three parameters, which equal to plane_normal * plane_offset. Suppose plane_normal is (a, b, c) and plane_offset is d, every point (X, Y, Z) on the plane satisfies, aX + bY + cZ = d. Then plane parameters are (a, b, c)*d. Since plane normal is a unit vector, we can extract plane_normal and plane_offset from their multiplication.\n \n## Run the inference code with an example - verified\n```bash\npython evaluate_original.py --methods=f --suffix=warping_refine --dataset=inference --customDataFolder=example_images\n```\n\nResults are saved under \"test/inference/\". Besides visualizations, plane parameters (#planes x 3) are saved in \"\\*_plane_parameters_0.npy\" and plane masks (#planes x 480 x 640) are saved in \"\\*_plane_masks_0.npy\".\n\n## Run the inference code with custom data - verified\nPlease put your images (*.png* or *.jpg* files), and camera intrinsics under a folder ($YOUR_IMAGE_FOLDER). The camera parameters should be put under a *.txt* file with 6 values (fx, fy, cx, cy, image_width, image_height) separately by a space. If the camera intrinsics is the same for all images, please put these parameters in *camera.txt*. Otherwise, please add a separate intrinsics file for each image, and name it the same with the image (changing the file extension to *.txt*). And then run:\n\ninstall IP Webcam app in your mobile,start server and give the ip address at 698th line \n\n```bash\npython evaluate_realtime.py --methods=f --suffix=warping_refine --dataset=inference \n```\n\n## Training - verified and training\n\n### Training data preparation\nPlease first download the ScanNet dataset (v2), unzip it to \"$ROOT_FOLDER/scans/\", and extract image frames from the *.sens* file using the official [reader](https://github.com/ScanNet/ScanNet/blob/master/SensReader/python/reader.py).\n\nThen download our plane annotation from [here](https://www.dropbox.com/s/u2wl4ji700u4shq/ScanNet_planes.zip?dl=0), and merge the \"scans/\" folder with \"$ROOT_FOLDER/scans/\". (If you prefer other locations, please change the paths in *datasets/scannet_scene.py*.)\n\nAfter the above steps, ground truth plane annotations are stored under \"$ROOT_FOLDER/scans/scene*/annotation/\". Among the annotations, *planes.npy* stores the plane parameters which are represented in the global frame. Plane segmentation for each image view is stored under *segmentation/*.\n\nTo generate such training data on your own, please refer to *data_prep/parse.py*. Please refer to the README under *data_prep/* for compilation.\n\nBesides scene-specific annotation under each scene folder, please download global metadata from [here](https://www.dropbox.com/s/v7qb7hwas1j766r/metadata.zip?dl=0), and unzip it to \"$ROOT_FOLDER\". Metadata includes the normal anchors (anchor_planes_N.npy) and invalid image indices caused by tracking issues (invalid_indices_*.txt). \n\nBackup gdrive link - plane annotations - scannet_5_scenes.zip - [here](https://drive.google.com/drive/folders/1b4yVyLvxknQ8e4yj5Jrw4r3Py9kj8hjf?usp=sharing)\n                     Global metadatas - metadata.zip - [here](https://drive.google.com/drive/folders/1EBDY0defytfDcqQvinufjTSe3I5rS7sT?usp=sharing)\n\n### Setting up dataloader for training - will add soon\n\nplease refer tree.md\n\n\n### Training script for scannet -verified\n\n```bash\npython train_planercnn.py --restore=1 --suffix=warping_refine --dataFolder=data_prep/Data/\n```\noptions:\n```bash\n--restore:\n- 0: training from scratch (not tested)\n- 1 (default): resume training from saved checkpoint\n- 2: training from pre-trained mask-rcnn model\n\n--suffix (the below arguments can be concatenated):\n- '': training the basic version\n- 'warping': with the warping loss\n- 'refine': with the refinement network\n- 'refine_only': train only the refinement work\n- 'warping_refine_after': add the warping loss after the refinement network instead of appending both independently\n\n--anchorType:\n- 'normal' (default): regress normal using 7 anchors\n- 'normal[k]' (e.g., normal5): regress normal using k anchors, normal0 will regress normal directly without anchors\n- 'joint': regress final plane parameters directly instead of predicting normals and depthmap separately\n```\n\nTemporary results are written under *test/* for debugging purposes.\n\n### Setting up Custom dataset(dataloader) for training - will add soon\n\nplease refer tree_custom_data.md\n\n\n### Training with custom data using scanent template of dataloader(pending)\n\nPlease refer the tree.md file and prepare dataset \n\n\nTo train on custom data, you need a list of planes, where each plane is represented using three parameters (as explained above) and a 2D binary mask. In our implementation, we use one 2D segmentation map where pixels with value *i* belong to the *i*th plane in the list. The easiest way is to replace the ScanNetScene class with something interacts with your custom data. Note that, the plane_info, which stores some semantic information and global plane index in the scene, is not used in this project. The code is misleading as global plane indices are read from plane_info [here](https://github.com/NVlabs/planercnn/blob/01e03fe5a97b7afc4c5c4c3090ddc9da41c071bd/datasets/plane_stereo_dataset.py#L194), but they are used only for debugging purposes.\n\n### Status note\n\n1. pending - still i didnt work on that part\n2. Working - I am working on that part\n3. Completed/Verified - its completely working and verified\n\n\n### Training script for custom dataset - Ajith is working in it not verified\n\n```bash\npython train_planercnn_custom.py --restore=1 --suffix=warping_refine --dataFolder=data_prep/Data/\n``` \n\n\n## Evaluation - verified\n\nTo evaluate the performance against existing methods, please run:\n```bash\npython evaluate.py --methods=f --suffix=warping_refine\n```\nOptions:\n```bash\n--methods:\n- f: evaluate PlaneRCNN (use --suffix and --anchorType to specify configuration as explained above)\n- p: evaluate PlaneNet\n- e: evaluate PlaneRecover\n- t: evaluate MWS (--suffix=gt for MWS-G)\n```\n### Observation\n\nA. Scannet Training from checkpoint(given in repo) for other scannet images - completed  \n    It was observered that we need to set the depth error to 0.3 (within limits) so that further training takes place without loss and there will be no \"nan\" and breaking of training.\n    Was able to train for 17 epcohs from pretrained checkpoint\n    Was able to inference the images after training\n\nGetting annotations from parse.py: -verified/completed\n        1. Go to [planenet](https://github.com/art-programmer/PlaneNet) set a conda env with python2.7 tensorflow==1.13.0 , opencv\n            - do everything in data_preparation/readme.md( install Openmesh from here https://github.com/TheWebMonks/meshmonk/blob/master/docs/ubuntu.md#installing-openmesh if you need)\n            - while running parse.py change 696th line to this cmd = './Renderer/Renderer --scene_id=' + scene_id + ' --root_folder=' + ROOT_FOLDER \n            - now you can get segmentation, plane_info.npy, planes.npy\n\nB. Scannet Training from scratch - (working)\n    It was observered that while training from scratch and depth error should be within 0.3 to keep the losses minimum, how ever during wrong initalization some times while training from scratch i have observered heavy losses. But mosty if depth error is controlled it will train without failing. However i trained only for certain epochs(4 or 5) so was not able to inference the images beacuase it was not generating planeXYZ for few epochs.\n    (One of the reason is dataset of scannet is large and downloading it and structuring even for 5 files will take more time. Also because of depth error in some images its skipping those images)\n  \n\nC. Proper method of annotating and constructing custom dataset - (working)\n    it was observered that plane info,segmentation,depth images is a important while training . So need to find methods to properly annotate and construct the info in accordance with the parameters in\n    planercnn\n    \n   method to genrate clean depth images similar to scannet or redwoods dataset - (pending)\n\n   it seems that we need a stero camera to genrate such perfect point cloud depth map. so i have took from [redwood indoor reconstruction dataset](http://redwood-data.org/indoor/dataset.html)\n    Able to Genrate custom segmentations without point cloud with this reprository - https://github.com/chaowang15/RGBDPlaneDetection\n    plane info is given by the repo\n    \n\nD. Custom images Training from checkpoint - (pending)\n\nE. Custom images Training from Scratch - (working)\n\n### Info about point cloud:\n\n1. https://github.com/jeffdelmerico/pointcloud_tutorial\n\n2. Converting rgbd to point cloud - install open3d - http://www.open3d.org/docs/latest/tutorial/Basic/rgbd_image.html - its really a beutifull one, 2d will be converted to 3d Wow!\n        dataset - http://redwood-data.org/indoor/dataset.html \n \n\n\n\n### Todo:\n\n1. Change dataloader for custom dataset - not completed - 27/11/2020  - working for from scratch training.\n2. Giving high loss as i have removed plane_info and anchors, need to replace properly and reduce the loss - 29/11/2020\n3. Need to fetch seperate planes for each image - 28/11/2020\n\n\n\n\n### Neighbour repos:\n\n\n### License ###\nCopyright (c) 2018 NVIDIA Corp.  All Rights Reserved.\nThis work is licensed under the [Creative Commons Attribution NonCommercial ShareAlike 4.0 License](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).\n\n\n## Contact\nIf you have any questions, please contact Ajith (mailto:inocajith21.5@gmail.com)\n\n## Acknowledgement\n\n1. The nms/roialign from the Mask R-CNN implementation from [pytorch-mask-rcnn](https://github.com/multimodallearning/pytorch-mask-rcnn), which is licensed under [MIT License](https://github.com/multimodallearning/pytorch-mask-rcnn/blob/master/LICENSE)\n2. The School of AI(https://theschoolof.ai/) and EVA5 students\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajithvcoder%2Fplanercnn_customimages","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fajithvcoder%2Fplanercnn_customimages","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajithvcoder%2Fplanercnn_customimages/lists"}