{"id":13444160,"url":"https://github.com/yangyanli/PointCNN","last_synced_at":"2025-03-20T18:31:15.866Z","repository":{"id":42425437,"uuid":"101113923","full_name":"yangyanli/PointCNN","owner":"yangyanli","description":"PointCNN: Convolution On X-Transformed Points (NeurIPS 2018)","archived":false,"fork":false,"pushed_at":"2021-09-03T08:13:02.000Z","size":343,"stargazers_count":1396,"open_issues_count":59,"forks_count":367,"subscribers_count":55,"default_branch":"master","last_synced_at":"2025-03-18T07:29:35.535Z","etag":null,"topics":["autonomous-driving","classification","convolutional-neural-networks","deep-neural-networks","machine-learning","point-cloud","pointcloud","robotics","scannet","segmentation","shapenet"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1801.07791","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yangyanli.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-08-22T22:45:32.000Z","updated_at":"2025-03-17T00:30:41.000Z","dependencies_parsed_at":"2022-08-24T02:50:34.842Z","dependency_job_id":null,"html_url":"https://github.com/yangyanli/PointCNN","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yangyanli%2FPointCNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yangyanli%2FPointCNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yangyanli%2FPointCNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yangyanli%2FPointCNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yangyanli","download_url":"https://codeload.github.com/yangyanli/PointCNN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244670067,"owners_count":20490916,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autonomous-driving","classification","convolutional-neural-networks","deep-neural-networks","machine-learning","point-cloud","pointcloud","robotics","scannet","segmentation","shapenet"],"created_at":"2024-07-31T03:02:20.615Z","updated_at":"2025-03-20T18:31:15.597Z","avatar_url":"https://github.com/yangyanli.png","language":"Python","readme":"# PointCNN: Convolution On X-Transformed Points\n\nCreated by \u003ca href=\"http://yangyan.li\" target=\"_blank\"\u003eYangyan Li\u003c/a\u003e, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and \u003ca href=\"https://cfcs.pku.edu.cn/baoquan/\" target=\"_blank\"\u003eBaoquan Chen\u003c/a\u003e.\n\n## Introduction\n\nPointCNN is a simple and general framework for feature learning from point cloud, which refreshed five benchmark records in point cloud processing (as of Jan. 23, 2018), including:\n\n* classification accuracy on ModelNet40 (**91.7%**, with 1024 input points only)\n* classification accuracy on ScanNet (**77.9%**)\n* segmentation part averaged IoU on ShapeNet Parts (**86.13%**)\n* segmentation mean IoU on S3DIS (**65.39%**)\n* per voxel labelling accuracy on ScanNet (**85.1%**)\n\nSee our \u003ca href=\"http://arxiv.org/abs/1801.07791\" target=\"_blank\"\u003epreprint on arXiv\u003c/a\u003e (accepted to NeurIPS 2018) for more details.\n\nPretrained models can be downloaded from \u003ca href=\"https://1drv.ms/f/s!AiHh4BK32df6gYFCzzpRz0nsJmQxSg\" target=\"_blank\"\u003ehere\u003c/a\u003e.\n\n### Performance on Recent Benchmarks\n\u003ca href=\"https://hkust-vgd.github.io/scanobjectnn/\" target=\"_blank\"\u003eRevisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data\u003c/a\u003e\n\n\u003ca href=\"https://arxiv.org/abs/1812.02713\" target=\"_blank\"\u003ePartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding\u003c/a\u003e\n\n\u003ca href=\"https://arxiv.org/abs/1812.06216\" target=\"_blank\"\u003eABC: A Big CAD Model Dataset For Geometric Deep Learning\u003c/a\u003e\n\n### Practical Applications\n\u003ca href=\"https://medium.com/geoai/3d-cities-deep-learning-in-three-dimensional-space-29f9dafdfd73\" target=\"_blank\"\u003e3D cities: Deep Learning in three-dimensional space\u003c/a\u003e (from \u003ca href=\"https://www.esri.com/en-us/home\" target=\"_blank\"\u003eEsri\u003c/a\u003e)\n\n\u003ca href=\"https://medium.com/geoai/pointcnn-replacing-50-000-man-hours-with-ai-d7397c1e7ffe\" target=\"_blank\"\u003ePointCNN: replacing 50,000 man hours with AI\u003c/a\u003e (from \u003ca href=\"https://www.esri.com/en-us/home\" target=\"_blank\"\u003eEsri\u003c/a\u003e)\n\n\u003ca href=\"https://developers.arcgis.com/python/guide/point-cloud-segmentation-using-pointcnn/\" target=\"_blank\"\u003e\nPoint Cloud Segmentation using PointCNN in ArcGIS API for Python\u003c/a\u003e (from \u003ca href=\"https://www.esri.com/en-us/home\" target=\"_blank\"\u003eEsri\u003c/a\u003e)\n\n### More Implementations\n* \u003ca href=\"https://github.com/rusty1s/pytorch_geometric\" target=\"_blank\"\u003ePytorch implementation from PyTorch Geometric\u003c/a\u003e\n* \u003ca href=\"https://github.com/hxdengBerkeley/PointCNN.Pytorch\" target=\"_blank\"\u003ePytorch implementation from Berkeley CS294-131 Course Proj\u003c/a\u003e\n* \u003ca href=\"https://github.com/chinakook/PointCNN.MX\" target=\"_blank\"\u003eMXNet implementation\u003c/a\u003e\n* \u003ca href=\"https://github.com/Jittor/PointCloudLib\" target=\"_blank\"\u003eJittor implementation\u003c/a\u003e\n\n**We highly welcome issues, rather than emails, for PointCNN related questions.**\n\n## License\nOur code is released under MIT License (see LICENSE file for details).\n\n## Code Organization\nThe core X-Conv and PointCNN architecture are defined in [pointcnn.py](pointcnn.py).\n\nThe network/training/data augmentation hyper parameters for classification tasks are defined in [pointcnn_cls](pointcnn_cls), for segmentation tasks are defined in [pointcnn_seg](pointcnn_seg).\n\n### Explanation of X-Conv and X-DeConv Parameters\nTake the xconv_params and xdconv_params from [shapenet_x8_2048_fps.py](pointcnn_seg/shapenet_x8_2048_fps.py) for example:\n```\nxconv_param_name = ('K', 'D', 'P', 'C', 'links')\nxconv_params = [dict(zip(xconv_param_name, xconv_param)) for xconv_param in\n                [(8, 1, -1, 32 * x, []),\n                 (12, 2, 768, 32 * x, []),\n                 (16, 2, 384, 64 * x, []),\n                 (16, 6, 128, 128 * x, [])]]\n\nxdconv_param_name = ('K', 'D', 'pts_layer_idx', 'qrs_layer_idx')\nxdconv_params = [dict(zip(xdconv_param_name, xdconv_param)) for xdconv_param in\n                 [(16, 6, 3, 2),\n                  (12, 6, 2, 1),\n                  (8, 6, 1, 0),\n                  (8, 4, 0, 0)]]\n```\nEach element in xconv_params is a tuple of (K, D, P, C, links), where K is the neighborhood size, D is the dilation rate, P is the representative point number in the output (-1 means all input points are output representative points), and C is the output channel number. The links are used for adding DenseNet style links, e.g., [-1, -2] will tell the current layer to receive inputs from the previous two layers. Each element specifies the parameters of one X-Conv layer, and they are stacked to create a deep network.\n\nEach element in xdconv_params is a tuple of (K, D, pts_layer_idx, qrs_layer_idx), where K and D have the same meaning as that in xconv_params, pts_layer_idx specifies the output of which X-Conv layer (from the xconv_params) will be the input of this X-DeConv layer, and qrs_layer_idx specifies the output of which X-Conv layer (from the xconv_params) will be forwarded and fused with the output of this X-DeConv layer. The P and C parameters of this X-DeConv layer is also determined by qrs_layer_idx. Similarly, each element specifies the parameters of one X-DeConv layer, and they are stacked to create a deep network.\n\n\n## PointCNN Usage\n\nPointCNN is implemented and tested with Tensorflow 1.6 in python3 scripts. **Tensorflow before 1.5 version is not recommended, because of API.** It has dependencies on some python packages such as transforms3d, h5py, plyfile, and maybe more if it complains. Install these packages before the use of PointCNN.\n\nIf you can only use Tensorflow 1.5 because of OS factor(UBUNTU 14.04),please modify \"isnan()\" to \"std::nan()\" in \"/usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/numeric_types.h\" line 49\n\nHere we list the commands for training/evaluating PointCNN on classification and segmentation tasks on multiple datasets.\n\n* ### Classification\n\n  * #### ModelNet40\n  ```\n  cd data_conversions\n  python3 ./download_datasets.py -d modelnet\n  cd ../pointcnn_cls\n  ./train_val_modelnet.sh -g 0 -x modelnet_x3_l4\n  ```\n\n  * #### ScanNet\n  Please refer to \u003chttp://www.scan-net.org/\u003e  for downloading ScanNet task data and scannet_labelmap, and refer to https://github.com/ScanNet/ScanNet/tree/master/Tasks/Benchmark for downloading ScanNet benchmark files:\n\n  scannet_dataset_download\n\n  |_ data\n\n  |_ scannet_labelmap\n\n  |_ benchmark\n\n  ```\n  cd ../data/scannet/scannet_dataset_download/\n  mv ./scannet_labelmap/scannet-labels.combined.tsv ../benchmark/\n\n  #./pointcnn_root\n  cd ../../../pointcnn/data_conversions\n  python extract_scannet_objs.py -f ../../data/scannet/scannet_dataset_download/data/ -b ../../data/scannet/scannet_dataset_download/benchmark/ -o ../../data/scannet/cls/\n  python prepare_scannet_cls_data.py -f ../../data/scannet/cls/\n  cd ../pointcnn_cls/\n  ./train_val_scannet.sh -g 0 -x scannet_x3_l4\n  ```\n\n  * #### tu_berlin\n  ```\n  cd data_conversions\n  python3 ./download_datasets.py -d tu_berlin\n  python3 ./prepare_tu_berlin_data.py -f ../../data/tu_berlin/ -a --create-train-test\n  cd ../pointcnn_cls\n  ./train_val_tu_berlin.sh -g 0 -x tu_berlin_x3_l4\n  ```\n\n  * #### quick_draw\n  Note that the training/evaluation of quick_draw requires LARGE RAM, as we load all stokes into RAM and converting them into point cloud on-the-fly.\n  ```\n  cd data_conversions\n  python3 ./download_datasets.py -d quick_draw\n  cd ../pointcnn_cls\n  ./train_val_quick_draw.sh -g 0 -x quick_draw_full_x2_l6\n  ```\n\n  * #### MNIST\n  ```\n  cd data_conversions\n  python3 ./download_datasets.py -d mnist\n  python3 ./prepare_mnist_data.py -f ../../data/mnist\n  cd ../pointcnn_cls\n  ./train_val_mnist.sh -g 0 -x mnist_x2_l4\n  ```\n\n  * #### CIFAR-10\n  ```\n  cd data_conversions\n  python3 ./download_datasets.py -d cifar10\n  python3 ./prepare_cifar10_data.py\n  cd ../pointcnn_cls\n  ./train_val_cifar10.sh -g 0 -x cifar10_x3_l4\n  ```\n\n* ### Segmentation\n\n  We use farthest point sampling (the implementation from \u003ca href=\"https://github.com/charlesq34/pointnet2\" target=\"_blank\"\u003ePointNet++\u003c/a\u003e) in segmentation tasks. Compile FPS before the training/evaluation:\n  ```\n  cd sampling\n  bash tf_sampling_compile.sh\n  ```\n\n  * #### ShapeNet\n  ```\n  cd data_conversions\n  python3 ./download_datasets.py -d shapenet_partseg\n  python3 ./prepare_partseg_data.py -f ../../data/shapenet_partseg\n  cd ../pointcnn_seg\n  ./train_val_shapenet.sh -g 0 -x shapenet_x8_2048_fps\n  ./test_shapenet.sh -g 0 -x shapenet_x8_2048_fps -l ../../models/seg/pointcnn_seg_shapenet_x8_2048_fps_xxxx/ckpts/iter-xxxxx -r 10\n  cd ../evaluation\n  python3 eval_shapenet_seg.py -g ../../data/shapenet_partseg/test_label -p ../../data/shapenet_partseg/test_data_pred_10 -a\n  ```\n\n  * #### S3DIS\n  Please refer to [data_conversions](data_conversions/README.md) for downloading S3DIS, then:\n  ```\n  cd data_conversions\n  python3 prepare_s3dis_label.py\n  python3 prepare_s3dis_data.py\n  python3 prepare_s3dis_filelists.py\n  mv S3DIS_files/* ../../data/S3DIS/out_part_rgb/\n  ./train_val_s3dis.sh -g 0 -x s3dis_x8_2048_fps -a 1\n  ./test_s3dis.sh -g 0 -x s3dis_x8_2048_fps -a 1 -l ../../models/seg/s3dis_x8_2048_fps_xxxx/ckpts/iter-xxxxx -r 4\n  cd ../evaluation\n  python3 s3dis_merge.py -d \u003cpath to *_pred.h5\u003e\n  python3 eval_s3dis.py\n  ```\n We use a hidden marker file to note when prepare is finished to avoid re-processing. This cache can be invalidated by deleting the markers. \n \n Please notice that these command just for Area 1 (specified by -a 1 option) validation. Results on other Areas can be computed by iterating -a option.\n\n  * #### ScanNet\n  Please refer to [data_conversions](data_conversions/README.md) for downloading ScanNet, then:\n  ```\n  cd data_conversions\n  python3 prepare_scannet_seg_data.py\n  python3 prepare_scannet_seg_filelists.py\n  cd ../pointcnn_seg\n  ./train_val_scannet.sh -g 0 -x scannet_x8_2048_k8_fps\n  ./test_scannet.sh -g 0 -x scannet_x8_2048_k8_fps -l ../../models/seg/pointcnn_seg_scannet_x8_2048_k8_fps_xxxx/ckpts/iter-xxxxx -r 4\n  cd ../evaluation\n  python3 eval_scannet.py -d \u003cpath to *_pred.h5\u003e -p \u003cpath to scannet_test.pickle\u003e\n  ```\n  * #### Semantic3D\n\n  Please check the free disk space before start, about 900 GB will be required.\n\n  ```\n  cd data_conversions\n  bash download_semantic3d.sh\n  bash un7z_semantic3d.sh\n  python3 prepare_semantic3d_data.py\n  mkdir ../../data/semantic3d/filelists\n  python3 prepare_semantic3d_filelists.py\n  cd ../pointcnn_seg\n  ./train_val_semantic3d.sh -g 0 -x semantic3d_x4_2048_fps\n  ./test_semantic3d.sh -g 0 -x semantic3d_x4_2048_fps -l \u003cpath to ckpt\u003e\n  cd ../evaluation\n  python3 semantic3d_merge.py -d \u003cpath to *_pred.h5\u003e -v \u003creduced or full\u003e\n  ```\n\n* ### Tensorboard\n  If you want to monitor your train step, we recommend you use the following command\n  ```\n  cd \u003cyour path\u003e/PointCNN\n  tensorboard --logdir=../models/\u003cseg/cls\u003e \u003c--port=6006\u003e\n  ```\n","funding_links":[],"categories":["Python","Sensor Processing","3DVision"],"sub_categories":["Lidar and Point Cloud Processing","3D SemanticSeg"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyangyanli%2FPointCNN","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyangyanli%2FPointCNN","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyangyanli%2FPointCNN/lists"}