{"id":22366572,"url":"https://github.com/microsoft/sparenet","last_synced_at":"2025-07-30T17:32:04.218Z","repository":{"id":53269335,"uuid":"343417962","full_name":"microsoft/SpareNet","owner":"microsoft","description":"Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021)","archived":false,"fork":false,"pushed_at":"2023-04-18T21:04:59.000Z","size":61736,"stargazers_count":145,"open_issues_count":16,"forks_count":20,"subscribers_count":9,"default_branch":"main","last_synced_at":"2024-12-04T17:49:06.876Z","etag":null,"topics":["deep-learning","differentiable-renderer","generative-adversarial-network","neural-network","point-cloud","point-cloud-completion"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2103.02535","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/microsoft.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null}},"created_at":"2021-03-01T13:01:29.000Z","updated_at":"2024-10-29T07:20:43.000Z","dependencies_parsed_at":"2022-08-27T06:40:21.571Z","dependency_job_id":"49a43bb3-6993-4aa6-897f-789ae975d29a","html_url":"https://github.com/microsoft/SpareNet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FSpareNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FSpareNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FSpareNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FSpareNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/microsoft","download_url":"https://codeload.github.com/microsoft/SpareNet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228164535,"owners_count":17879085,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","differentiable-renderer","generative-adversarial-network","neural-network","point-cloud","point-cloud-completion"],"created_at":"2024-12-04T18:14:09.520Z","updated_at":"2024-12-04T18:14:11.240Z","avatar_url":"https://github.com/microsoft.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021)\n\nAn efficient PyTorch library for Point Cloud Completion. \n\n\u003c!-- ![image](./teaser.png) --\u003e\n\u003cdiv  align=\"center\"\u003e    \n\u003cimg src=\"./teaser.png\" width = \"800\"   align=center /\u003e\n\u003c/div\u003e\n\n### [Project page](https://alphapav.github.io/SpareNet/) |   [Paper](https://arxiv.org/abs/2103.02535) | [Video]()\n\n[Chulin Xie*](https://github.com/AlphaPav), [Chuxin Wang*](https://chuxwa.github.io/), [Bo Zhang](https://bo-zhang.me/), [Hao Yang](https://www.microsoft.com/en-us/research/people/haya/), [Dong Chen](https://www.microsoft.com/en-us/research/people/doch/), and [Fang Wen](https://www.microsoft.com/en-us/research/people/fangwen/). (\\*Equal contribution)\n\n## Abstract\n\u003eWe proposed a novel Style-based Point Generator with Adversarial Rendering (SpareNet) for point cloud completion. Firstly, we present the channel-attentive EdgeConv to fully exploit the local structures as well as the global shape in point features. Secondly, we observe that the concatenation manner used by vanilla foldings limits its potential of generating a complex and faithful shape. Enlightened by the success of StyleGAN, we regard the shape feature as style code that modulates the normalization layers during the folding, which considerably enhances its capability. Thirdly, we realize that existing point supervisions, e.g., Chamfer Distance or Earth Mover’s Distance, cannot faithfully reflect the perceptual quality of the reconstructed points. To address this, we propose to project the completed points to depth maps with a differentiable renderer and apply adversarial training to advocate the perceptual realism under different viewpoints. Comprehensive experiments on ShapeNet and KITTI prove the effectiveness of our method, which achieves state-of-the-art quantitative performance while offering superior visual quality.\n\n\n## Installation\n\n1. Create a virtual environment via `conda`.\n\n   ```shell\n   conda create -n sparenet python=3.7\n   conda activate sparenet\n   ```\n\n2. Install `torch` and `torchvision`.\n\n   ```shell\n   conda install pytorch cudatoolkit=10.1 torchvision -c pytorch\n   ```\n\n3. Install requirements.\n\n   ```shell\n   pip install -r requirements.txt\n   ```\n\n4. Install cuda\n   ```shell\n   sh setup_env.sh\n   ```\n\n\n## Dataset\n* Download [the processed ShapeNet dataset](https://gateway.infinitescript.com/?fileName=ShapeNetCompletion) (16384 points) generated by [GRNet](https://github.com/hzxie/GRNet), and the [KITTI dataset](https://drive.google.com/drive/folders/1fSu0_huWhticAlzLh3Ejpg8zxzqO1z-F). \n\n* Update the file path of the datasets in `configs/base_config.py`:\n\n   ```\n   __C.DATASETS.shapenet.partial_points_path = \"/path/to/datasets/ShapeNetCompletion/%s/partial/%s/%s/%02d.pcd\"\n   __C.DATASETS.shapenet.complete_points_path = \"/path/to/datasets/ShapeNetCompletion/%s/complete/%s/%s.pcd\"\n   __C.DATASETS.kitti.partial_points_path = \"/path/to/datasets/KITTI/cars/%s.pcd\"\n   __C.DATASETS.kitti.bounding_box_file_path = \"/path/to/datasets/KITTI/bboxes/%s.txt\"\n\n   # Dataset Options: ShapeNet, ShapeNetCars, KITTI\n   __C.DATASET.train_dataset = \"ShapeNet\"\n   __C.DATASET.test_dataset = \"ShapeNet\"\n   ```\n\n\n## Get Started\n\n### Inference Using Pretrained Model\n\nThe pretrained models:\n\n- [SpareNet for ShapeNet](https://drive.google.com/file/d/15PiH-bRlSlK4AUUnVwREzuAlMVJ9TfQG) \n- [PCN for ShapeNet](https://drive.google.com/drive/folders/1ruN16MlJm4OeRMd41C19HyWqYOIrNrNh)\n- [GRNet for ShapeNet](https://gateway.infinitescript.com/?fileName=GRNet-ShapeNet.pth) \n- [GRNet for KITTI](https://gateway.infinitescript.com/?fileName=GRNet-KITTI.pth) \n- [MSN for ShapeNet](https://drive.google.com/drive/folders/14UZXKqXIZ0gL3hhrV2ySll_pH2eLGFL5) (for 8192 points)\n\n\n-  run\n\n   ```shell\n   python test.py --gpu ${GPUS}\\\n            --workdir ${WORK_DIR} \\\n            --model ${network} \\\n            --weights ${path to checkpoint} \\\n            --test_mode ${mode}\n   ```\n\n-  example\n   ```shell\n   python  test.py --gpu 0 --workdir /path/to/logfiles --model sparenet --weights /path/to/checkpoint --test_mode default\n   ```\n\n### Train\n\nAll log files in the training process, such as log message, checkpoints, etc, will be saved to the work directory.\n\n-  run\n\n   ```shell\n   python train.py  --gpu ${GPUS}\\\n            --workdir ${WORK_DIR} \\\n            --model ${network} \\\n            --weights ${path to checkpoint}\n   ```\n-  example\n   ```shell\n   python  train.py --gpu 0,1,2,3 --workdir /path/to/logfiles --model sparenet --weights /path/to/checkpoint\n   ```\n\n\n\n## Differentiable Renderer\nA fully differentiable point renderer that enables end-to-end rendering from 3D point cloud to 2D depth maps. See the paper for details.\n\n\n\u003cdiv  align=\"center\"\u003e    \n\u003cimg src=\"./renderer.png\" width = \"600\"   align=center /\u003e\n\u003c/div\u003e\n\n\n\n### Usage of Renderer\n\nThe inputs of renderer are pcd, views and radius, and the outputs of renderer are depth_maps.\n-  example\n   ```shell\n   # `projection_mode`: a str with value \"perspective\" or \"orthorgonal\"\n   # `eyepos_scale`: a float that defines the distance of eyes to (0, 0, 0)\n   # `image_size`: an int defining the output image size\n   renderer = ComputeDepthMaps(projection_mode, eyepos_scale, image_size)\n\n   # `data`: a tensor with shape [batch_size, num_points, 3]\n   # `view_id`: the index of selected view satisfying 0 \u003c= view_id \u003c 8\n   # `radius_list`: a list of floats, defining the kernel radius to render each point\n   depthmaps = renderer(data, view_id, radius_list)\n   ```\n\n### Test FPD on ShapeNet Dataset\n* Run your model and save your results of test dataset\n\n* Update the file path of the results in `test_fpd.py` and run it:\n   ```\n   parser.add_argument('--log_dir', default='/path/to/save/logs')\n   parser.add_argument('--data_dir', default='/path/to/test/dataset/pcds')\n   parser.add_argument('--fake_dir', default='/path/to/methods/pcds',\n                              help='/path/to/results/shapenet_fc/pcds/')\n   ```\n\n## License\n\nThe codes and the pretrained model in this repository are under the MIT license as specified by the LICENSE file. \n\nThis project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.\n\n\n## BibTex\n\nIf you like our work and use the codebase or models for your research, please cite our work as follows.\n\n```bibtex\n@InProceedings{Xie_2021_CVPR,\n    author    = {Xie, Chulin and Wang, Chuxin and Zhang, Bo and Yang, Hao and Chen, Dong and Wen, Fang},\n    title     = {Style-Based Point Generator With Adversarial Rendering for Point Cloud Completion},\n    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n    month     = {June},\n    year      = {2021},\n    pages     = {4619-4628}\n}\n``` \n\n\u003c!-- ## Acknowledgement\nWe thank for the inspiration from [MSN]() and [GRNet]() --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2Fsparenet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmicrosoft%2Fsparenet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2Fsparenet/lists"}