{"id":23082443,"url":"https://github.com/humanpose1/ms-svconv","last_synced_at":"2025-08-16T00:31:26.628Z","repository":{"id":43062311,"uuid":"351507725","full_name":"humanpose1/MS-SVConv","owner":"humanpose1","description":"Compute descriptors for 3D point cloud registration using a multi scale sparse voxel architecture ","archived":false,"fork":false,"pushed_at":"2022-03-21T10:56:16.000Z","size":3245,"stargazers_count":63,"open_issues_count":4,"forks_count":7,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-04-04T02:01:31.102Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/humanpose1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-03-25T16:42:16.000Z","updated_at":"2025-01-22T09:29:46.000Z","dependencies_parsed_at":"2022-09-03T16:00:11.236Z","dependency_job_id":null,"html_url":"https://github.com/humanpose1/MS-SVConv","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/humanpose1/MS-SVConv","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/humanpose1%2FMS-SVConv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/humanpose1%2FMS-SVConv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/humanpose1%2FMS-SVConv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/humanpose1%2FMS-SVConv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/humanpose1","download_url":"https://codeload.github.com/humanpose1/MS-SVConv/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/humanpose1%2FMS-SVConv/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270651049,"owners_count":24622434,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-15T02:00:12.559Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-16T14:52:45.656Z","updated_at":"2025-08-16T00:31:26.310Z","avatar_url":"https://github.com/humanpose1.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 3D Point Cloud Registration with Multi-Scale Architecture and Unsupervised Transfer Learning\n\nCompute features for 3D point cloud registration. The article is available on [Arxiv](https://arxiv.org/abs/2103.14533). This article has been accepted in [3DV2021](https://3dv2021.surrey.ac.uk/accepted-papers/).\n \n \n \u003cimg src=\"image/overview.png\"  height=\"600\"\u003e\n\n\nIt relies on:\n- MS-SVConv : A multi scale sparse voxel architecture\n- UDGE: Unsupervised transfer learning using Data GEneration\nThe combination of both allows better generalization capabilities and transfer across different datasets.\n\nThe code is also available on the [torch-points3d repository](https://github.com/nicolas-chaulet/torch-points3d).\nThis repository is to show how to launch the code for training and testing.\n\n## Demo\nIf you want to try MS-SVConv without installing anything on your computer, A Google colab notebook is available [here](https://colab.research.google.com/github/humanpose1/MS-SVConv/blob/main/notebook/demo_MSSVConv.ipynb) (it takes few minutes to install everything). In the colab, we compute features using MS-SVConv and use Ransac (implementation of Open3D) to compute the transformation.\nYou can try on 3DMatch on ETH. With this notebook, you can directly use the pretrained model on your project !\nThe code have been tried on an NVDIA RTX 1080 Ti with CUDA version 10.1 and Ubuntu 18.04.\n\n\n## Train and test the model using the library torch-points3d\n\n### Installation for training and evaluation\nThis installation step is necessary if you want to train and evaluate MS-SVConv.\n\n\nfirst you need, to clone the [torch-points3d repository](https://github.com/nicolas-chaulet/torch-points3d)\n```\ngit clone https://github.com/nicolas-chaulet/torch-points3d.git\n```\nTorch-points3d uses [poetry](https://python-poetry.org/) to manage the packages. after installing Poetry, run :\n```\npoetry install --no-root\n```\nActivate the environnement\n```\npoetry shell\n```\nIf you want to train MS-SVConv on 3DMatch, you will need pycuda (It's optional for testing and for other dataset).\n```\npip install pycuda\n```\nYou will also need to install [Minkowski Engine](https://github.com/NVIDIA/MinkowskiEngine) and [torchsparse](https://github.com/mit-han-lab/torchsparse)\nFinally, you will need [TEASER++](https://github.com/MIT-SPARK/TEASER-plusplus) for testing.\n\nIf you have problems with installation (espaecially with [pytorch_geometric](https://github.com/rusty1s/pytorch_geometric)), please visit the [Troubleshooting section](https://github.com/nicolas-chaulet/torch-points3d#troubleshooting) of torch-points3d page.\n\n## Training\n\n### registration\n\n\nIf you want to train MS-SVConv with 3 heads starting at the scale 2cm, run this command:\n```\npoetry run python train.py task=registration models=registration/ms_svconv_base model_name=MS_SVCONV_B2cm_X2_3head data=registration/fragment3dmatch training=sparse_fragment_reg tracker_options.make_submission=True training.epochs=200 eval_frequency=10\n```\n\nautomatically, the code will call the right yaml file in `conf/data/registration` for the dataset and `conf/model/registration` for the model.\nIf you just want to train MS-SVConv with 1 head, run this command\n```\npoetry run python train.py task=registration models=registration/ms_svconv_base model_name=MS_SVCONV_B2cm_X2_1head data=registration/fragment3dmatch training=sparse_fragment_reg tracker_options.make_submission=True training.epochs=200 eval_frequency=10\n```\nYou can modify some hyperparameters directly on the command line. For example, if you want to change the learning rate of `1e-2`, you can run:\n```\npoetry run python train.py task=registration models=registration/ms_svconv_base model_name=MS_SVCONV_B2cm_X2_1head data=registration/fragment3dmatch training=sparse_fragment_reg tracker_options.make_submission=True training.epochs=200 eval_frequency=10 training.optim.base_lr=1e-2\n```\n\nTo resume training:\n```\npoetry run python train.py task=registration models=registration/ms_svconv_base model_name=MS_SVCONV_B2cm_X2_3head data=registration/fragment3dmatch training=sparse_fragment_reg tracker_options.make_submission=True epochs=200 eval_frequency=10 training.checkpoint_dir=/path/of/directory/containing/pretrained/model\n```\n\n\nWARNING : On 3DMatch, you will need a lot of disk space because the code will download the RGBD image on 3DMatch and build the fragments from scratch. Also the code takes time (few hours).\n\n\nFor 3DMatch, it was supervised training because the pose is necessary. But we can also fine-tune in a self-supervised fashion (without needing the pose).\n\nTo train on Modelnet run this command:\n```\npoetry run python train.py task=registration models=registration/ms_svconv_base model_name=MS_SVCONV_B2cm_X2_3head data=registration/modelnet_sparse_ss training=sparse_fragment_reg tracker_options.make_submission=True training.epochs=200 eval_frequency=10\n```\n\nTo fine-tune on ETH run this command (First, download the pretrained model from 3DMatch [here](https://cloud.mines-paristech.fr/index.php/s/hRc6y2YIFtYsGAI/)):\n```\npoetry run python train.py task=registration models=registration/ms_svconv_base model_name=MS_SVCONV_B4cm_X2_3head data=registration/eth_base training=sparse_fragment_reg_finetune tracker_options.make_submission=True training.epochs=200 eval_frequency=10 models.path_pretrained=/path/to/your/pretrained/model.pt\n```\nTo fine-tune on TUM, run this command:\n```\npoetry run python train.py task=registration models=registration/ms_svconv_base model_name=MS_SVCONV_B4cm_X2_3head data=registration/testtum_ss training=sparse_fragment_reg_finetune tracker_options.make_submission=True training.epochs=200 eval_frequency=10 models.path_pretrained=/path/to/your/pretrained/model.pt\n```\n\n\nFor all these command, it will save in `outputs` directory log of the training, it will save a `.pt` file which is the weights of\n\n### semantic segmentation\n\nYou can also train MS-SVConv on scannet for semantic segmentation. To do this simply run:\n```\npoetry run python train.py task=segmentation models=segmentation/ms_svconv_base model_name=MS_SVCONV_B4cm_X2_3head lr_scheduler.params.gamma=0.9922 data=segmentation/scannet-sparse training=minkowski_scannet tracker_options.make_submission=False tracker_options.full_res=False data.process_workers=1 training.wandb.log=True eval_frequency=10 training.batch_size=4\n```\nAnd you can easily transfer from registration to segmantation, with this command:\n```\npoetry run python train.py task=segmentation models=segmentation/ms_svconv_base model_name=MS_SVCONV_B4cm_X2_3head lr_scheduler.params.gamma=0.9922 data=segmentation/scannet-sparse training=minkowski_scannet tracker_options.make_submission=False tracker_options.full_res=False data.process_workers=1 training.wandb.log=True eval_frequency=10 batch_size=4 models.path_pretrained=/path/to/your/pretrained/model.pt\n```\n\n## Evaluation\n\nIf you want to evaluate the models on 3DMatch,  download the model [here](https://cloud.mines-paristech.fr/index.php/s/m93S2jx91GRrduu) and run:\n\n```\npoetry run python scripts/test_registration_scripts/evaluate.py task=registration models=registration/ms_svconv_base model_name=MS_SVCONV_B2cm_X2_3head data=registration/fragment3dmatch training=sparse_fragment_reg cuda=True data.sym=True training.checkpoint_dir=/directory/of/the/models/\n```\non ETH (model [here](https://cloud.mines-paristech.fr/index.php/s/pUmGPtHUG2ASxlJ)),\n```\npoetry run python scripts/test_registration_scripts/evaluate.py task=registration models=registration/ms_svconv_base model_name=MS_SVCONV_B4cm_X2_3head data=registration/eth_base training=sparse_fragment_reg cuda=True data.sym=True training.checkpoint_dir=/directory/of/the/models/\n```\non TUM (model [here](https://cloud.mines-paristech.fr/index.php/s/LUVEZvutaDpEkVD)),\n```\npoetry run python scripts/test_registration_scripts/evaluate.py task=registration models=registration/ms_svconv_base model_name=MS_SVCONV_B2cm_X2_3head data=registration/testtum_ss training=sparse_fragment_reg cuda=True data.sym=True training.checkpoint_dir=/directory/of/the/models/\n```\nYou can also visualize matches, you can run:\n```\npython scripts/test_registration_scripts/see_matches.py task=registration models=registration/ms_svconv_base model_name=MS_SVCONV_B4cm_X2_3head data=registration/eth_base training=sparse_fragment_reg cuda=True data.sym=True training.checkpoint_dir=/directory/of/the/models/ data.first_subsampling=0.04 +ind=548 +t=22\n```\n\nYou should obtain this image\n \u003cimg src=\"image/match.png\"  height=\"600\"\u003e\n \n## Small Updates\n\nWith the current version of torch-points3d, we obtain a FMR of 97.4 insted of 98.4 with `tau=0.05` on 3DMatch in a supervised setting. However, with `tau=0.2`, we obtain a FMR of 91.6 instead of 89.9. In other word, the new version is better in terms of FMR with `tau=0.2`.\nIf you want to get the previous results, you can use the previous version.\n```\ngit clone https://github.com/humanpose1/deeppointcloud-benchmarks.git\ngit checkout -b \"MS_SVCONV_B2cm_X2_3head\" d079374da05506762f32bb7b090f35be86a90760\n```\nYou will have to install older versions of omegaconf, hydra, torch and torchsparse. \nthe model can be downloaded [here](https://cloud.mines-paristech.fr/index.php/s/kFGIqiY9Ky5OeiB)\n\n\n## Model Zoo\nYou can find all the pretrained model  (More will be added in the future)\n\n## citation\nIf you like our work, please cite it :\n```\n@inproceedings{horache2021mssvconv,\nauthor = {S. Horache and J. Deschaud and F. Goulette},\nbooktitle = {2021 International Conference on 3D Vision (3DV)},\ntitle = {3D Point Cloud Registration with Multi-Scale Architecture and Unsupervised Transfer Learning},\nyear = {2021},\nvolume = {},\nissn = {},\npages = {1351-1361},\ndoi = {10.1109/3DV53792.2021.00142},\nurl = {https://doi.ieeecomputersociety.org/10.1109/3DV53792.2021.00142},\npublisher = {IEEE Computer Society},\naddress = {Los Alamitos, CA, USA},\nmonth = {dec}\n}\n}\n```\nAnd if you use ETH, 3DMatch, TUM or ModelNet as dataset, please cite the respective authors.\n\n## TODO\n - Add other pretrained models on the model zoo\n - Add others datasets such as KITTI Dataset\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhumanpose1%2Fms-svconv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhumanpose1%2Fms-svconv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhumanpose1%2Fms-svconv/lists"}