{"id":13443566,"url":"https://github.com/astra-vision/LMSCNet","last_synced_at":"2025-03-20T16:31:56.543Z","repository":{"id":38822978,"uuid":"303411227","full_name":"astra-vision/LMSCNet","owner":"astra-vision","description":"LMSCNet: Lightweight Multiscale 3D Semantic Completion. Roldão, L., de Charette, R., \u0026 Verroust-Blondet, A. International Conference on 3D Vision (3DV). 2020","archived":false,"fork":false,"pushed_at":"2022-06-02T13:31:34.000Z","size":949,"stargazers_count":81,"open_issues_count":2,"forks_count":15,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-28T06:58:32.343Z","etag":null,"topics":["3d-reconstruction","computer-vision","reconstruction","semantickitti"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/astra-vision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-12T14:06:05.000Z","updated_at":"2024-07-31T09:07:03.000Z","dependencies_parsed_at":"2022-08-26T20:02:42.398Z","dependency_job_id":null,"html_url":"https://github.com/astra-vision/LMSCNet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astra-vision%2FLMSCNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astra-vision%2FLMSCNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astra-vision%2FLMSCNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astra-vision%2FLMSCNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/astra-vision","download_url":"https://codeload.github.com/astra-vision/LMSCNet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244649936,"owners_count":20487522,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-reconstruction","computer-vision","reconstruction","semantickitti"],"created_at":"2024-07-31T03:02:03.944Z","updated_at":"2025-03-20T16:31:56.174Z","avatar_url":"https://github.com/astra-vision.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# LMSCNet: Lightweight Multiscale 3D Semantic Completion\nOfficial repository.  \n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/lmscnet-lightweight-multiscale-3d-semantic/3d-semantic-scene-completion-on-semantickitti)](https://paperswithcode.com/sota/3d-semantic-scene-completion-on-semantickitti?p=lmscnet-lightweight-multiscale-3d-semantic)\n\n## Paper\n![alt text](teaser.png \"LMSCNet\")\n\n[LMSCNet: Lightweight Multiscale 3D Semantic Completion](https://arxiv.org/abs/2008.10559) \\\n[Luis Roldão](https://team.inria.fr/rits/membres/luis-roldao-jimenez/), [Raoul de Charette](https://team.inria.fr/rits/membres/raoul-de-charette/), [Anne Verroust-Blondet](https://team.inria.fr/rits/membres/anne-verroust/)  \nInria, Akka Research. 3DV 2020 (oral) \\\n[[Demo Video]](https://www.youtube.com/watch?v=J6dYoWx4Xqw\u0026feature=youtu.be)\n\n\nIf you find our work useful, please cite:\n```\n@inproceedings{roldao2020lmscnet,\n  title={LMSCNet: Lightweight Multiscale 3D Semantic Completion},\n  author={Rold{\\~a}o, Luis and de Charette, Raoul and Verroust-Blondet, Anne},\n  booktitle={International Conference on 3D Vision (3DV)},\n  year={2020}\n}\n```\n\n## Preparation\n### Prerequisites\nTested with\n* PyTorch 1.3.1\n* CUDA 10.2\n* Python 3.7.5\n* Numpy 1.17.4\n\n### Setup\nWe advise to create a new conda environment for installation.\n\n```\n$ conda create --name lmscnet_ssc python=3.7.5 numpy tqdm scipy scikit-learn pyyaml imageio tensorboard -y\n$ conda activate lmscnet_ssc\n$ conda install pytorch torchvision cudatoolkit=10.0 -c pytorch\n```\n\nThen clone this repository in desired location\n```\n$ git clone https://github.com/cv-rits/LMSCNet\n```\n\n### Dataset\n\nPlease download the Full Semantic Scene Completion dataset (v1.1) from the [SemanticKITII website](http://www.semantic-kitti.org/dataset.html) and extract it.\n\nYou need to preprocess the data to generate lower scale labels for LMSCNet first.\nThe preprocessing performs majority pooling over high-resolution original scale label\ngrids (1:1) in order to obtain ground-truth data at lower resolutions (1:2, 1:4 and 1:8).\nIt also generates validity masks as such resolutions to consider the loss on known voxels only,\nas in original scale data. All information will be stored in the same format and respective location\nthan the semanticKITTI provided data with new file extensions (`file.label_1_X` and `file.invalid_1_X`).\n\nIf you are using **v1.1** of the dataset, you can download the data directly from [here](https://www.rocq.inria.fr/rits_files/download.php?file=computer-vision/lmscnet/semanticKITTI_v1.1_dscale.zip).\nPlease extract the data into the semanticKITTI root folder. \n\nOtherwise, you need to generate the data by running the `LMSCNet/data/labels_downscale.py` as follows:\n```\n$ cd \u003croot dir of this repo\u003e\n$ python LMSCNet/data/labels_downscale.py --dset_root \u003cpath/dataset/root\u003e\n```\n* `dset_root` should point to the root directory of the SemanticKITTI dataset (containing `dataset` folder)\n\n## Training\n\nAll training settings can be edited by using the yaml file generator in `SSC_configs/config_routine.py`. We provide training\nroutine examples in the `SSC_configs/examples/` folder. Make sure to change the dataset path to your extracted dataset location in such files if you \nwant to use them for training. Additionally, you can change the folder where the performance and stats will be stored, this folder has been defined as `SSC_out` by default.\n* `config_dict['DATASET']['ROOT_DIR']` should be changed to the root directory of the SemanticKITTI dataset (containing `dataset` folder)\n* `config_dict['OUTPUT']['OUT_ROOT'] ` to be changed to desired output folder.\n\n### LMSCNet \u0026 LMSCNet-SS\nYou can run the training with\n```\n$ cd \u003croot dir of this repo\u003e\n$ python LMSCNet/train.py --cfg SSC_configs/examples/LMSCNet.yaml --dset_root \u003cpath/dataset/root\u003e\n```\n\nWe also provide single scale version of our network which can achieve slightly better performance at\nthe cost of losing multiscale capacity:\n```\n$ cd \u003croot dir of this repo\u003e\n$ python LMSCNet/train.py --cfg SSC_configs/examples/LMSCNet_SS.yaml --dset_root \u003cpath/dataset/root\u003e\n```\n\n### Baselines\nTrain coded baselines with:\n```\n$ python LMSCNet/train.py --cfg SSC_configs/examples/SSCNet.yaml --dset_root \u003cpath/dataset/root\u003e\n$ python LMSCNet/train.py --cfg SSC_configs/examples/SSCNet_full.yaml --dset_root \u003cpath/dataset/root\u003e\n```\n\nIn all previous examples you need to provide your path to the dataset folder, if not provided, the path\nset in the `.yaml` file will be used by default.\n\n## Validating \u0026 Testing\n\nValidation passes are done during training routine. Additional pass in the validation set with saved model \nto check performance can be done by using the `LMSCNet/validate.py` file. You need to provide the path to the saved model and the \ndataset root directory.\n\n```\n$ cd \u003croot dir of this repo\u003e\n$ python LMSCNet/validate.py --weights \u003c/path/to/model.pth\u003e --dset_root \u003cpath/dataset/root\u003e\n```\n\nSince SemantiKITTI contains a hidden test set, we provide test routine to save predicted output in same\nformat of SemantiKITTI, which can be compressed and uploaded to the [SemanticKITTI Semantic Scene Completion Benchmark](http://www.semantic-kitti.org/tasks.html#semseg).\nWe recommend to pass compressed data through official checking script provided in the [SemanticKITTI Development Kit](http://www.semantic-kitti.org/resources.html#devkit) to avoid any issue.\nYou can provide which checkpoints you want to use for testing. We used the ones that performed best on the validation set during training.\nYou need to provide the path to the saved model, the \ndataset root directory and the output path to where the predictions will be stored.\n\n```\n$ cd \u003croot dir of this repo\u003e\n$ python LMSCNet/test.py --weights \u003c/path/to/model.pth\u003e --dset_root \u003cpath/dataset/root\u003e --out_path \u003cpredictions/output/path\u003e\n```\n\n## Ablation\n\nWe test the robustness of our network against sparsity by retrieving the original 64-layers KITTI scans used in SemanticKITTI\nand subsampling 8/16/32 layers LiDARs with layers subsampling.\n\n_coming up soon..._\n\n\u003c!---\n## Acknowledgements\nThis work has been developed in collaboration between INRIA and AKKA Technologies, under the funding of AKKA Technologies. \n--\u003e\n\n## Model Zoo\n\nYou can download the models with the scores below from this [Google drive folder](https://drive.google.com/drive/folders/12A46LE3BO6tQ8Y5OFbR4ImP7nzM8_3wb?usp=sharing).\n\n| Method                    | SC IoU               | SSC mIoU             |\n| ------------------------- | -------------------- | -------------------- |\n| SSCNet-full               | 49.98\u003csup\u003e*\u003c/sup\u003e    | 16.14\u003csup\u003e*\u003c/sup\u003e    |\n| LMSCNet                   | 55.32\u003csup\u003e*\u003c/sup\u003e    | 17.01\u003csup\u003e*\u003c/sup\u003e    |\n| LMSCNet-SS                | 56.72\u003csup\u003e*\u003c/sup\u003e    | 17.62\u003csup\u003e*\u003c/sup\u003e    |\n\n\n\u003csup\u003e*\u003c/sup\u003e Results reported correspond to SemanticKITTI hidden test set V1.0. \nSemanticKITTI benchmark has recently changed\nto v1.1 due to grid shift issue ([link](https://github.com/PRBonn/semantic-kitti-api/issues/49)). This should bring also slight differences if re-uploaded to test benchmark. \n\n## License\nLMSCNet is released under the [Apache 2.0 license](./LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fastra-vision%2FLMSCNet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fastra-vision%2FLMSCNet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fastra-vision%2FLMSCNet/lists"}