{"id":13444061,"url":"https://github.com/lyqun/FPConv","last_synced_at":"2025-03-20T17:33:02.962Z","repository":{"id":54790017,"uuid":"247041055","full_name":"lyqun/FPConv","owner":"lyqun","description":"CVPR 2020, \"FPConv: Learning Local Flattening for Point Convolution\"","archived":false,"fork":false,"pushed_at":"2021-11-03T15:12:49.000Z","size":1608,"stargazers_count":131,"open_issues_count":5,"forks_count":16,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-10-28T07:42:38.711Z","etag":null,"topics":["3d-convolutions","3d-vision","cvpr2020","point-cloud","point-convolution","pytorch","s3dis","scannet","scene-understanding","semantic-segmentation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lyqun.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-13T10:05:24.000Z","updated_at":"2024-09-10T02:12:17.000Z","dependencies_parsed_at":"2022-08-14T03:00:53.796Z","dependency_job_id":null,"html_url":"https://github.com/lyqun/FPConv","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyqun%2FFPConv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyqun%2FFPConv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyqun%2FFPConv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyqun%2FFPConv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lyqun","download_url":"https://codeload.github.com/lyqun/FPConv/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244660952,"owners_count":20489427,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-convolutions","3d-vision","cvpr2020","point-cloud","point-convolution","pytorch","s3dis","scannet","scene-understanding","semantic-segmentation"],"created_at":"2024-07-31T03:02:17.960Z","updated_at":"2025-03-20T17:33:02.461Z","avatar_url":"https://github.com/lyqun.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# FPConv\n\nYiqun Lin, Zizheng Yan, Haibin Huang, Dong Du, Ligang Liu, Shuguang Cui, Xiaoguang Han, \"FPConv: Learning Local Flattening for Point Convolution\", CVPR 2020 [[paper]](https://arxiv.org/abs/2002.10701)\n\n```\n@inproceedings{lin2020fpconv,\n  title={Fpconv: Learning local flattening for point convolution},\n  author={Lin, Yiqun and Yan, Zizheng and Huang, Haibin and Du, Dong and Liu, Ligang and Cui, Shuguang and Han, Xiaoguang},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={4293--4302},\n  year={2020}\n}\n```\n\n## Introduction\n\nWe introduce FPConv, a novel surface-style convolution operator designed for 3D point cloud analysis. Unlike previous methods, FPConv doesn't require transforming to intermediate representation like 3D grid or graph and directly works on surface geometry of point cloud. To be more specific, for each point, FPConv performs a local flattening by automatically learning a weight map to softly project surrounding points onto a 2D grid. Regular 2D convolution can thus be applied for efficient feature learning. FPConv can be easily integrated into various network architectures for tasks like 3D object classification and 3D scene segmentation, and achieve comparable performance with existing volumetric-type convolutions. More importantly, our experiments also show that FPConv can be a complementary of volumetric convolutions and jointly training them can further boost overall performance into state-of-the-art results.\n\n![fig3](figures/fig3.jpg)\n\n## Installation\n\nThis code has been tested with Python 3.6, PyTorch 1.2.0, CUDA 10.0 and CUDNN 7.4 on Ubuntu 18.04. \n\nFirstly, install [pointnet2](https://github.com/sshaoshuai/Pointnet2.PyTorch) by running the following commands:\n\n```shell\ncd fpconv/pointnet2\npython setup.py install\ncd ../\n```\n\nYou may also need to install plyfile and pickle for data preprocessing.\n\n## Usage\n\nEdit the global configuration file `config.json` before training.\n\n```json\n{\n    \"version\": \"0.0\",\n    \"scannet_raw\": \"\u003cpath_to_this_repo\u003e/dataset/scannet_v2\",\n    \"scannet_pickle\": \"\u003cpath_to_this_repo\u003e/dataset/scannet_pickles\",\n    \"scene_list\": \"\u003cpath_to_this_repo\u003e/utils/scannet_datalist\",\n    \"s3dis_aligned_raw\": \"\u003cpath_to_this_repo\u003e/dataset/Stanford3dDataset_v1.2_Aligned_Version\",\n    \"s3dis_data_root\": \"\u003cpath_to_this_repo\u003e/dataset/s3dis_aligned\"\n}\n```\n\n### — Semantic Segmentation on ScanNet\n\n__0. Baseline__\n\nTested on eval split of ScanNet. (see `./utils/scannet_datalist/scannetv2_eval.txt`)\n\n| Model         | mIoU | mA   | oA   | download                                                     |\n| ------------- | ---- | ---- | ---- | ------------------------------------------------------------ |\n| fpcnn_scannet | 64.4 | 76.4 | 85.8 | [ckpt-17.8M](https://drive.google.com/file/d/1jR-m3bx2tGo9oV4ULdaYr-woSiel659T/view?usp=sharing) |\n\n__1. Preprocessing__\n\nDownload ScanNet v2 dataset to `./dataset/scannet_v2`. Only `_vh_clean_2.ply` and `_vh_clean_2.labels.ply` should be downloaded for each scene. Specify the dataset path and output path in `config.json` and then run following commands for data pre-processing.\n\n```shell\ncd utils\npython collect_scannet_pickle.py\n```\n\nIt will generate 3 pickle files (`scannet_\u003csplit\u003e_rgb21c_pointid.pickle`) for 3 splits (train, eval, test) respectively. We also provide a pre-processed ScanNet v2 dataset for downloading: [Google Drive](https://drive.google.com/drive/folders/1xz59bKaIZbf0BU3oKSTs3qyV3gRf7aDW?usp=sharing). The `./dataset` folder should be organized as follows.\n\n```\nFPConv\n├── dataset\n│   ├── scannet_v2\n│   │  ├── scans\n│   │  ├── scans_test\n│   ├── scannet_pickles\n│   │  ├── scannet_train_rgb21c_pointid.pickle\n│   │  ├── scannet_eval_rgb21c_pointid.pickle\n│   │  ├── scannet_test_rgb21c_pointid.pickle\n```\n\n__2. Training__\n\nRun the following command to start the training. Output (logs) will be redirected to `./logs/fp_scannet/nohup.log`.\n\n```shell\nbash train_scannet.sh\n```\n\nWe trained our model with 2 Titan Xp GPUs with batch size of 12. If you don't have enough GPUs for training, please reduce `batch_size` to 6 for single GPU.\n\n__3. Evaluation__\n\nRun the following command to evaluate model on evaluation dataset (you may need to modify the `epoch` in `./test_scannet.sh`). Output (logs) will be redirected to `./test/fp_scannet_240.log`.\n\n```shell\nbash test_scannet.sh\n```\n\n__Note__: Final evaluation (by running `./test_scannet.sh`) is conducted on full point cloud, while evaluation during the training phase is conducted on randomly sampled points in each block of input scene.\n\n### — Semantic Segmentation on S3DIS\n\n__0. Baseline__\n\nTrained on Area 1~4 and 6, tested on Area 5.\n\n| Model       | mIoU | mA   | oA   | download                                                     |\n| ----------- | ---- | ---- | ---- | ------------------------------------------------------------ |\n| fpcnn_s3dis | 62.7 | 70.3 | 87.5 | [ckpt-70.0M](https://drive.google.com/file/d/1v5FHDYPfcji3elUQJ-P6n618EZ7_2rpd/view?usp=sharing) |\n\n__1. Preprocessing__\n\nFirstly, you need to download S3DIS dataset from [link](http://buildingparser.stanford.edu/dataset.html), aligned version 1.2 of the dataset is used in this work. Unzip S3DIS dataset into `./dataset/Stanford3dDataset_v1.2_Aligned_Version`, and specify the dataset path and output path in `config.json`. Then run the following commands for pre-processing.\n\n```shell\ncd utils\npython collect_indoor3d_data.py\ncd ..\n```\n\nIt will generate a  `.npy` files for each room. The dataset folder should be organized as follows. We also provide a pre-processed S3DIS dataset for downloading: [Google Drive](https://drive.google.com/file/d/1Pdf8x-Ayz8n5YxkouU1R9nD71LEctAtq/view?usp=sharing).\n\n```\nFPConv\n├── dataset\n│   ├── Stanford3dDataset_v1.2_Aligned_Version\n│   │  ├── Area_1\n│   │  ├── ...\n│   │  ├── Area_6\n│   ├── s3dis_aligned\n│   │  ├── *.npy\n```\n\n__2. Training__\n\nRun the following command to start the training. Output (logs) will be redirected to `./logs/fp_s3dis/nohup.log`.\n\n```shell\nbash train_s3dis.sh\n```\n\nWe trained our model on S3DIS with 4 Titan Xp GPUs, batch size of 8 totally, and 100 epochs.\n\n__3. Evaluation__\n\nRun the following command to evaluate model on evaluation dataset (you may need to modify the `epoch` in `./test_s3dis.sh`). Output (logs) will be redirected to `./test/fp_s3dis_60.log`.\n\n```shell\nbash test_s3dis.sh\n```\n\n## Acknowledgement\n\n- [sshaoshuai/Pointnet2.PyTorch](https://github.com/sshaoshuai/Pointnet2.PyTorch): PyTorch implementation of PointNet++.\n- [charlesq34/pointnet](https://github.com/charlesq34/pointnet): Data pre-processing for S3DIS.\n- [DylanWusee/pointconv](https://github.com/DylanWusee/pointconv): Data pre-processing for ScanNet.\n\n## License\n\nThis repository is released under MIT License (see LICENSE file for details).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flyqun%2FFPConv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flyqun%2FFPConv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flyqun%2FFPConv/lists"}