{"id":14537634,"url":"https://github.com/prs-eth/PanopticSegForLargeScalePointCloud","last_synced_at":"2025-09-02T20:30:57.979Z","repository":{"id":179054461,"uuid":"662459889","full_name":"prs-eth/PanopticSegForLargeScalePointCloud","owner":"prs-eth","description":"Code and dataset of paper \"TOWARDS ACCURATE INSTANCE SEGMENTATION IN LARGE-SCALE LIDAR POINT CLOUDS\"","archived":false,"fork":false,"pushed_at":"2024-11-04T13:51:36.000Z","size":10147,"stargazers_count":78,"open_issues_count":13,"forks_count":11,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-12-25T06:10:01.283Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prs-eth.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-05T07:26:23.000Z","updated_at":"2024-12-09T10:28:05.000Z","dependencies_parsed_at":"2023-07-07T11:45:50.965Z","dependency_job_id":"e0f30d94-cc75-41aa-804d-129e51d74479","html_url":"https://github.com/prs-eth/PanopticSegForLargeScalePointCloud","commit_stats":null,"previous_names":["bxiang233/panopticsegforlargescalepointcloud","prs-eth/panopticsegforlargescalepointcloud"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FPanopticSegForLargeScalePointCloud","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FPanopticSegForLargeScalePointCloud/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FPanopticSegForLargeScalePointCloud/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FPanopticSegForLargeScalePointCloud/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prs-eth","download_url":"https://codeload.github.com/prs-eth/PanopticSegForLargeScalePointCloud/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231806556,"owners_count":18429390,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-05T10:01:06.447Z","updated_at":"2024-12-30T02:30:43.089Z","avatar_url":"https://github.com/prs-eth.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Towards Accurate Instance Segmentation in Large-scale LiDAR Point Clouds\n\nThis repository represents the official code for paper entitled \"Towards accurate instance segmentation in large-scale LiDAR point clouds\". This study explores the steps of the panoptic segmentation pipeline concerned with clustering points into object instances, with the goal to alleviate that bottleneck. We find that a carefully designed clustering strategy, which leverages multiple types of learned point embeddings, significantly improves instance segmentation. Experiments on the NPM3D urban mobile mapping dataset and the FOR-instance forest dataset demonstrate the effectiveness and versatility of the proposed strategy.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"100%\" src=\"/Docs/instance%20segmentation%20visualization%20new.png\" /\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"100%\" src=\"/Docs/instance%20segmentation%20visualization2.png\" /\u003e\n\u003c/p\u003e\n\n# Set up environment\n\nThe framework used in this code is torchpoints-3d, so generally the installation instructions for torchpoints-3d can follow the official ones: \n\nhttps://torch-points3d.readthedocs.io/en/latest/\n\nhttps://github.com/torch-points3d/torch-points3d\n\nHere are two detailed examples for installation worked on our local computers for your reference:\n\n### Example 1 of installation\n\nSpecs local computer: Ubuntu 22.04, 64-bit, CUDA version 11.7 -\u003e but CUDA is backwards compatible, so here we used CUDA 11.1 for all libraries installed.\n\nCommands in terminal using miniconda:\n```bash\nconda create -n treeins_env_local python=3.8\n\nconda activate treeins_env_local\n\nconda install pytorch=1.9.0 torchvision==0.10.0 torchaudio==0.9.0 cudatoolkit=11.1 -c pytorch -c nvidia\n\npip install numpy==1.19.5\n\nconda install openblas-devel -c anaconda\n\nexport CUDA_HOME=/usr/local/cuda-11\n\npip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option=\"--blas_include_dirs=${CONDA_PREFIX}/include\" –install-option=\"--blas=openblas\"\n\n#CHECK IF TORCH AND MINKOWSKI ENGINE WORK AS EXPECTED:\n(treeins_env_local) : python\nPython 3.8.13 (default, #DATE#, #TIME#)\n[GCC 7.5.0] :: Anaconda, Inc. on linux\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n\u003e\u003e\u003e import torch\n\u003e\u003e\u003e torch.cuda.is_available()\nTrue\n\u003e\u003e\u003e import MinkowskiEngine\n\u003e\u003e\u003e exit()\n#CHECK FINISHED\n\npip install torch-scatter==2.0.8 -f https://data.pyg.org/whl/torch-1.9.0+cu111.html\n\npip install torch-sparse==0.6.12 -f https://data.pyg.org/whl/torch-1.9.0+cu111.html\n\npip install torch-geometric==1.7.2\n\n#We got the file requirements.txt from here: https://github.com/nicolas-chaulet/torch-points3d/blob/master/requirements.txt but deleted the lines containing the following libraries in the file: torch, torchvision, torch-geometric,  torch-scatter, torch-sparse, numpy\npip install -r requirements.txt\n\npip install numba==0.55.1\n\nconda install -c conda-forge hdbscan==0.8.27\n\nconda install numpy-base==1.19.2\n\npip install joblib==1.1.0\n```\n\n### Example 2 of installation\nSpecs local computer: Ubuntu 22.04.1, 64-bit, CUDA version 11.3\n```bash\nconda create -n torchpoint3denv python=3.8\nconda activate torchpoint3denv\nconda install -c conda-forge gcc==9.4.0\nconda install pytorch=1.9.0 torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia\nconda install numpy==1.19.5\nmamba install libopenblas openblas\nfind ${CONDA_PREFIX}/include -name \"cblas.h\"\nexport CXX=g++\nexport MAX_JOBS=2;\ngit clone https://github.com/NVIDIA/MinkowskiEngine.git\ncd MinkowskiEngine\npython setup.py install --blas_include_dirs=${CONDA_PREFIX}/include --blas=openblas\n\n#THE STEPS START FROM HERE ARE EXACT THE SAME AS EXAMPLE 1 \n#CHECK IF TORCH AND MINKOWSKI ENGINE WORK AS EXPECTED: \n#...\n\n```\n\nBased on our experience, we would suggest build most of the packages from the source for a larger chance of succesful installation. Good luck to your installation!\n\n# Data introduction\n\n## NPM3D dataset with instance labels\nLink for dataset download:\nhttps://doi.org/10.5281/zenodo.8118986\n\n### Semantic labels\n1: \"ground\",\n2: \"buildings\",\n3: \"poles\",\n4: \"bollards\",\n5: \"trash cans\",\n6: \"barriers\",\n7: \"pedestrians\",\n8: \"cars\",\n9: \"natural\"\n\n### Data folder structure\n```bash\n├─ conf                    # All configurations for training and evaluation leave there\n├─ forward_scripts         # Script that runs a forward pass on possibly non annotated data\n├─ outputs                 # All outputs from your runs\n├─ scripts                 # Some scripts to help manage the project\n├─ torch_points3d\n├─ data                    # DATA FOLDER\n    └─ npm3dfused\n        └─ raw\n            ├─ *_train.ply          # All train files\n            ├─ *_val.ply            # All val files\n            └─ *_test.ply           # All test files\n├─ train.py                # Main script to launch a training\n└─ eval.py                 # Eval script\n```\n\n## FOR-instance dataset\nUpdated!! Since FOR-instance dataset is open. I open the link ((https://polybox.ethz.ch/index.php/s/wVBlHgH308GRr1c)) for download: \n1. all .ply files\n2. the pretrained model for the input radius of cylinders = 8m\n\n### Different forest regions/subgroups of trees\n|\u003ch3\u003e Forest region \u003c/h3\u003e|\u003ch3\u003e Number of files \u003ch3\u003e|\u003ch3\u003e Approx. largest tree heights \u003c/h3\u003e|\n| :--------------------: | :---------------------: | :-------------------------------: |\n| \u003ch3\u003e CULS \u003c/h3\u003e | 3 | 25|\n| \u003ch3\u003e NIBIO \u003c/h3\u003e | 20 | 30|\n| \u003ch3\u003e NIBIO2 \u003c/h3\u003e | 50 | 25|\n| \u003ch3\u003e RMIT \u003c/h3\u003e | 2 | 15|\n| \u003ch3\u003e SCION \u003c/h3\u003e | 5 | 30|\n| \u003ch3\u003e TUWIEN \u003c/h3\u003e | 2 | 35|\n\n### Train - validation - test split\nTrain - test split is given by NIBIO: 56 train files, 26 test files. We decided on choosing 25% of the train files randomly but fixed as validation set -\u003e 42 train files, 14 val files, 26 test files. \n\n### Data folder structure\n```bash\n├─ conf                    # All configurations for training and evaluation leave there\n├─ forward_scripts         # Script that runs a forward pass on possibly non annotated data\n├─ outputs                 # All outputs from your runs\n├─ scripts                 # Some scripts to help manage the project\n├─ torch_points3d\n├─ data                    # DATA FOLDER\n    └─ treeinsfused\n        └─ raw\n            ├─ CULS\n                ├─ *_train.ply          # All train files\n                ├─ *_val.ply            # All val files\n                └─ *_test.ply           # All test files\n            ├─ NIBIO\n                ├─ *_train.ply          # SIMILAR AS CULS FOLDER\n                ├─ *_val.ply            \n                └─ *_test.ply           \n            ├─ NIBIO2\n                ├─ *.ply                # SIMILAR AS CULS FOLDER          \n            ├─ RMIT\n                ├─ *.ply                # SIMILAR AS CULS FOLDER\n            ├─ SCION\n                ├─ *.ply                # SIMILAR AS CULS FOLDER\n            └─ TUWIEN\n                ├─ *.ply                # SIMILAR AS CULS FOLDER\n├─ train.py                # Main script to launch a training\n└─ eval.py                 # Eval script\n```\n\n# Getting started with code\n\n## How to set different parameters\n\n|\u003ch3\u003e Parameter \u003c/h3\u003e|\u003ch3\u003e Value \u003ch3\u003e|\u003ch3\u003e Where to find/How to set in code \u003c/h3\u003e|\n| :--------------------: | :---------------------: | :-------------------------------: |\n| Choose different settings in Table 2 in the original paper| Setting I-V| Setting I: models=panoptic/area4_ablation_19, Setting II: models=panoptic/area4_ablation_14, Setting III: models=panoptic/area4_ablation_15, Setting IV: models=panoptic/area4_ablation_3heads_5 Setting V: models=panoptic/area4_ablation_3heads_6 |\n| Number of training iterations | 150 epochs | conf/training/#NameOfYourChosenConfigFile#.yaml, epochs|\n| Voxel size/subsampling size | 0.2 (m) | conf/data/panoptic/#NameOfYourChosenConfigFile#.yaml, first_subsampling |\n| Radius of sampling cylinder | 8 (m) | conf/data/panoptic/#NameOfYourChosenConfigFile#.yaml, radius |\n| The folder name of your output files | #YourOutputFolderName# | job_name=#YourOutputFolderName# |\n\n1. Create wandb account and specify your own wandb account name in conf/training/*.yaml. Have a look at all needed configurations of your current run in conf/data/panoptic/*.yaml, conf/models/panoptic/*.yaml and conf/training/*.yaml. Perform training by running:\n\n```bash\n# An example for NPM3D dataset\n# Run Setting IV for test area 1, radius=16m, voxel side length=0.12m, training for 200 epoches.\npython train.py task=panoptic data=panoptic/npm3d-sparseconv_grid_012_R_16_cylinder_area1 models=panoptic/area4_ablation_3heads_5 model_name=PointGroup-PAPER training=7_area1 job_name=A1_S7\n\n# An example for FOR-instance dataset\n# Run Setting IV, radius=8m, voxel side length=0.2m, training for 150 epoches.\npython train.py task=panoptic data=panoptic/treeins_rad8 models=panoptic/area4_ablation_3heads_5 model_name=PointGroup-PAPER training=treeins job_name=treeins_my_first_run\n```\n\n2. Look at \"TO ADAPT\" comments in conf/eval.yaml and change accordingly. Perform evaluation by running:\n```bash\npython eval.py\n```\n\n3. Look at \"TO ADAPT\" comments in evaluation_stats.py and change accordingly, then run:\n```bash\n# For NPM3D dataset\npython evaluation_stats_NPM3D.py\n\n# For FOR-instance dataset\npython evaluation_stats_FOR.py\n```\n\n# Citing\nIf you find our work useful, please do not hesitate to cite it:\n\n```\n@inproceedings{\n  Xiang2023,\n  title={Towards Accurate Instance Segmentation in Large-scale LiDAR Point Clouds},\n  author={Binbin Xiang and Torben Peters and Theodora Kontogianni and Frawa Vetterli1 and Stefano Puliti and Rasmus Astrup and Konrad Schindler},\n  booktitle={2023 The ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},\n  year={2023},\n  url = {\\url{https://arxiv.org/abs/2307.02877}}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprs-eth%2FPanopticSegForLargeScalePointCloud","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprs-eth%2FPanopticSegForLargeScalePointCloud","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprs-eth%2FPanopticSegForLargeScalePointCloud/lists"}