{"id":13443498,"url":"https://github.com/Pang-Yatian/Point-MAE","last_synced_at":"2025-03-20T16:31:33.076Z","repository":{"id":37451897,"uuid":"467804363","full_name":"Pang-Yatian/Point-MAE","owner":"Pang-Yatian","description":"[ECCV2022] Masked Autoencoders for Point Cloud Self-supervised Learning","archived":false,"fork":false,"pushed_at":"2023-06-26T01:51:27.000Z","size":1409,"stargazers_count":428,"open_issues_count":0,"forks_count":53,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-08-01T03:43:47.761Z","etag":null,"topics":["point-cloud","self-supervised-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Pang-Yatian.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-03-09T06:26:31.000Z","updated_at":"2024-07-31T02:33:05.000Z","dependencies_parsed_at":"2023-01-29T15:30:32.999Z","dependency_job_id":null,"html_url":"https://github.com/Pang-Yatian/Point-MAE","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pang-Yatian%2FPoint-MAE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pang-Yatian%2FPoint-MAE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pang-Yatian%2FPoint-MAE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pang-Yatian%2FPoint-MAE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Pang-Yatian","download_url":"https://codeload.github.com/Pang-Yatian/Point-MAE/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221780014,"owners_count":16879040,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["point-cloud","self-supervised-learning"],"created_at":"2024-07-31T03:02:02.190Z","updated_at":"2024-10-28T04:31:10.668Z","avatar_url":"https://github.com/Pang-Yatian.png","language":"Python","funding_links":[],"categories":["Python","2022"],"sub_categories":[],"readme":"# Point-MAE\n\n## Masked Autoencoders for Point Cloud Self-supervised Learning, [ECCV 2022](https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136620591.pdf), [ArXiv](https://arxiv.org/abs/2203.06604)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/masked-autoencoders-for-point-cloud-self/3d-point-cloud-classification-on-scanobjectnn)](https://paperswithcode.com/sota/3d-point-cloud-classification-on-scanobjectnn?p=masked-autoencoders-for-point-cloud-self)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/masked-autoencoders-for-point-cloud-self/3d-point-cloud-classification-on-modelnet40)](https://paperswithcode.com/sota/3d-point-cloud-classification-on-modelnet40?p=masked-autoencoders-for-point-cloud-self)\n\nIn this work, we present a novel scheme of masked autoencoders for point cloud self-supervised learning, termed as Point-MAE. Our Point-MAE is neat and efficient, with minimal modifications based on the properties of the point cloud. In classification tasks, Point-MAE outperforms all the other self-supervised learning methods on ScanObjectNN and ModelNet40. Point-MAE also advances state-of-the-art accuracies by 1.5%-2.3% in the few-shot learning on ModelNet40. \n\n\u003cdiv  align=\"center\"\u003e    \n \u003cimg src=\"./figure/net.jpg\" width = \"666\"  align=center /\u003e\n\u003c/div\u003e\n\n## 1. Requirements\nPyTorch \u003e= 1.7.0 \u003c 1.11.0;\npython \u003e= 3.7;\nCUDA \u003e= 9.0;\nGCC \u003e= 4.9;\ntorchvision;\n\n```\npip install -r requirements.txt\n```\n\n```\n# Chamfer Distance \u0026 emd\ncd ./extensions/chamfer_dist\npython setup.py install --user\ncd ./extensions/emd\npython setup.py install --user\n# PointNet++\npip install \"git+https://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops\u0026subdirectory=pointnet2_ops_lib\"\n# GPU kNN\npip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl\n```\n\n## 2. Datasets\n\nWe use ShapeNet, ScanObjectNN, ModelNet40 and ShapeNetPart in this work. See [DATASET.md](./DATASET.md) for details.\n\n## 3. Point-MAE Models\n|  Task | Dataset | Config | Acc.| Download|      \n|  ----- | ----- |-----|  -----| -----|\n|  Pre-training | ShapeNet |[pretrain.yaml](./cfgs/pretrain.yaml)| N.A. | [here](https://github.com/Pang-Yatian/Point-MAE/releases/download/main/pretrain.pth) |\n|  Classification | ScanObjectNN |[finetune_scan_hardest.yaml](./cfgs/finetune_scan_hardest.yaml)| 85.18%| [here](https://github.com/Pang-Yatian/Point-MAE/releases/download/main/scan_hardest.pth)  |\n|  Classification | ScanObjectNN |[finetune_scan_objbg.yaml](./cfgs/finetune_scan_objbg.yaml)|90.02% | [here](https://github.com/Pang-Yatian/Point-MAE/releases/download/main/scan_objbg.pth) |\n|  Classification | ScanObjectNN |[finetune_scan_objonly.yaml](./cfgs/finetune_scan_objonly.yaml)| 88.29%| [here](https://github.com/Pang-Yatian/Point-MAE/releases/download/main/scan_objonly.pth) |\n|  Classification | ModelNet40(1k) |[finetune_modelnet.yaml](./cfgs/finetune_modelnet.yaml)| 93.80%| [here](https://github.com/Pang-Yatian/Point-MAE/releases/download/main/modelnet_1k.pth) |\n|  Classification | ModelNet40(8k) |[finetune_modelnet_8k.yaml](./cfgs/finetune_modelnet_8k.yaml)| 94.04%| [here](https://github.com/Pang-Yatian/Point-MAE/releases/download/main/modelnet_8k.pth) |\n| Part segmentation| ShapeNetPart| [segmentation](./segmentation)| 86.1% mIoU| [here](https://github.com/Pang-Yatian/Point-MAE/releases/download/main/part_seg.pth) |\n\n|  Task | Dataset | Config | 5w10s Acc. (%)| 5w20s Acc. (%)| 10w10s Acc. (%)| 10w20s Acc. (%)|     \n|  ----- | ----- |-----|  -----| -----|-----|-----|\n|  Few-shot learning | ModelNet40 |[fewshot.yaml](./cfgs/fewshot.yaml)| 96.3 ± 2.5| 97.8 ± 1.8| 92.6 ± 4.1| 95.0 ± 3.0| \n\n## 4. Point-MAE Pre-training\nTo pretrain Point-MAE on ShapeNet training set, run the following command. If you want to try different models or masking ratios etc., first create a new config file, and pass its path to --config.\n\n```\nCUDA_VISIBLE_DEVICES=\u003cGPU\u003e python main.py --config cfgs/pretrain.yaml --exp_name \u003coutput_file_name\u003e\n```\n## 5. Point-MAE Fine-tuning\n\nFine-tuning on ScanObjectNN, run:\n```\nCUDA_VISIBLE_DEVICES=\u003cGPUs\u003e python main.py --config cfgs/finetune_scan_hardest.yaml \\\n--finetune_model --exp_name \u003coutput_file_name\u003e --ckpts \u003cpath/to/pre-trained/model\u003e\n```\nFine-tuning on ModelNet40, run:\n```\nCUDA_VISIBLE_DEVICES=\u003cGPUs\u003e python main.py --config cfgs/finetune_modelnet.yaml \\\n--finetune_model --exp_name \u003coutput_file_name\u003e --ckpts \u003cpath/to/pre-trained/model\u003e\n```\nVoting on ModelNet40, run:\n```\nCUDA_VISIBLE_DEVICES=\u003cGPUs\u003e python main.py --test --config cfgs/finetune_modelnet.yaml \\\n--exp_name \u003coutput_file_name\u003e --ckpts \u003cpath/to/best/fine-tuned/model\u003e\n```\nFew-shot learning, run:\n```\nCUDA_VISIBLE_DEVICES=\u003cGPUs\u003e python main.py --config cfgs/fewshot.yaml --finetune_model \\\n--ckpts \u003cpath/to/pre-trained/model\u003e --exp_name \u003coutput_file_name\u003e --way \u003c5 or 10\u003e --shot \u003c10 or 20\u003e --fold \u003c0-9\u003e\n```\nPart segmentation on ShapeNetPart, run:\n```\ncd segmentation\npython main.py --ckpts \u003cpath/to/pre-trained/model\u003e --root path/to/data --learning_rate 0.0002 --epoch 300\n```\n\n## 6. Visualization\n\nVisulization of pre-trained model on ShapeNet validation set, run:\n\n```\npython main_vis.py --test --ckpts \u003cpath/to/pre-trained/model\u003e --config cfgs/pretrain.yaml --exp_name \u003cname\u003e\n```\n\n\u003cdiv  align=\"center\"\u003e    \n \u003cimg src=\"./figure/vvv.jpg\" width = \"900\"  align=center /\u003e\n\u003c/div\u003e\n\n## Acknowledgements\n\nOur codes are built upon [Point-BERT](https://github.com/lulutang0608/Point-BERT), [Pointnet2_PyTorch](https://github.com/erikwijmans/Pointnet2_PyTorch) and [Pointnet_Pointnet2_pytorch](https://github.com/yanx27/Pointnet_Pointnet2_pytorch)\n\n## Reference\n\n```\n@inproceedings{pang2022masked,\n  title={Masked autoencoders for point cloud self-supervised learning},\n  author={Pang, Yatian and Wang, Wenxiao and Tay, Francis EH and Liu, Wei and Tian, Yonghong and Yuan, Li},\n  booktitle={Computer Vision--ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part II},\n  pages={604--621},\n  year={2022},\n  organization={Springer}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPang-Yatian%2FPoint-MAE","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPang-Yatian%2FPoint-MAE","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPang-Yatian%2FPoint-MAE/lists"}