{"id":13442312,"url":"https://github.com/dvlab-research/Stratified-Transformer","last_synced_at":"2025-03-20T13:33:25.320Z","repository":{"id":37407782,"uuid":"471729233","full_name":"dvlab-research/Stratified-Transformer","owner":"dvlab-research","description":"Stratified Transformer for 3D Point Cloud Segmentation (CVPR 2022)","archived":false,"fork":false,"pushed_at":"2023-01-07T14:19:05.000Z","size":222,"stargazers_count":368,"open_issues_count":39,"forks_count":40,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-10-28T05:13:03.257Z","etag":null,"topics":["cvpr2022","point-cloud","semantic-segmentation","transformer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dvlab-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-03-19T15:04:55.000Z","updated_at":"2024-10-22T16:24:15.000Z","dependencies_parsed_at":"2023-02-07T09:46:21.703Z","dependency_job_id":null,"html_url":"https://github.com/dvlab-research/Stratified-Transformer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvlab-research%2FStratified-Transformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvlab-research%2FStratified-Transformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvlab-research%2FStratified-Transformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvlab-research%2FStratified-Transformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dvlab-research","download_url":"https://codeload.github.com/dvlab-research/Stratified-Transformer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244619279,"owners_count":20482392,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cvpr2022","point-cloud","semantic-segmentation","transformer"],"created_at":"2024-07-31T03:01:44.207Z","updated_at":"2025-03-20T13:33:24.992Z","avatar_url":"https://github.com/dvlab-research.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Stratified Transformer for 3D Point Cloud Segmentation\n*Xin Lai\u003csup\u003e\\*\u003c/sup\u003e, Jianhui Liu\u003csup\u003e\\*\u003c/sup\u003e, Li Jiang, Liwei Wang, Hengshuang Zhao, Shu Liu, Xiaojuan Qi, Jiaya Jia*\n\nThis is the official PyTorch implementation of our paper [**Stratified Transformer for 3D Point Cloud Segmentation**](https://arxiv.org/pdf/2203.14508.pdf) that has been accepted to CVPR 2022. [\\[arXiv\\]](https://arxiv.org/pdf/2203.14508.pdf) [\\[CVF\\]](https://openaccess.thecvf.com/content/CVPR2022/papers/Lai_Stratified_Transformer_for_3D_Point_Cloud_Segmentation_CVPR_2022_paper.pdf)\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"figs/fig.jpg\"/\u003e\n\u003c/div\u003e\n\n# Highlight \n1. Our method (*Stratified Transformer*) achieves the state-of-the-art performance on 3D point cloud semantic segmentation on both S3DIS and ScanNetv2 datasets. **It is the first time for a point-based method to outperform the voxel-based ones**, such as SparseConvNet and MinkowskiNet;\n2. *Stratified Transformer* is point-based, and constructed by Transformer with standard multi-head self-attention, enjoying large receptive field, robust generalization ability as well as competitive performance;\n3. This repository develops a memory-efficient implementation to combat the issue of **variant-length tokens** with several CUDA kernels, avoiding unnecessary momery occupation of vacant tokens. We also use shared memory for further acceleration.\n\n# Get Started\n\n## Environment\n\n1. Install dependencies\n\n```\npip install -r requirements.txt\n```\n\nIf you have any problem with the above command, you can also install them by\n\n```\npip install torch_sparse==0.6.12\npip install torch_points3d==1.3.0\npip install tensorboard timm termcolor tensorboardX\n```\n\n2. Compile pointops\n\nMake sure you have installed `gcc` and `cuda`, and `nvcc` can work (Note that if you install cuda by conda, it won't provide nvcc and you should install cuda manually.). Then, compile and install pointops2 as follows. (We have tested on gcc==7.5.0 and cuda==10.1)\n```\ncd lib/pointops2\npython3 setup.py install\n```\n\n## Datasets Preparation\n\n### S3DIS\nPlease refer to https://github.com/yanx27/Pointnet_Pointnet2_pytorch for S3DIS preprocessing. Then modify the `data_root` entry in the .yaml configuration file.\n\n### ScanNetv2\nPlease refer to https://github.com/dvlab-research/PointGroup for the ScanNetv2 preprocessing. Then change the `data_root` entry in the .yaml configuration file accordingly.\n\n## Training\n\n### S3DIS\n- Stratified Transformer\n```\npython3 train.py --config config/s3dis/s3dis_stratified_transformer.yaml\n```\n\n- 3DSwin Transformer (The vanilla version shown in our paper)\n```\npython3 train.py --config config/s3dis/s3dis_swin3d_transformer.yaml\n```\n\n### ScanNetv2\n- Stratified Transformer\n```\npython3 train.py --config config/scannetv2/scannetv2_stratified_transformer.yaml\n```\n\n- 3DSwin Transformer (The vanilla version shown in our paper)\n```\npython3 train.py --config config/scannetv2/scannetv2_swin3d_transformer.yaml\n```\n\nNote: It is normal to see the the results on S3DIS fluctuate between -0.5\\% and +0.5\\% mIoU maybe because the size of S3DIS is relatively small, while the results on ScanNetv2 are relatively stable.\n\n## Testing\nFor testing, first change the `model_path`, `save_folder` and `data_root_val` (if applicable) accordingly. Then, run the following command. \n```\npython3 test.py --config [YOUR_CONFIG_PATH]\n```\n\n## Pre-trained Models\n\nFor your convenience, you can download the pre-trained models and training/testing logs from [Here](https://mycuhk-my.sharepoint.com/:f:/g/personal/1155154502_link_cuhk_edu_hk/EihXWr_HEnJIvR_M0_YRbSgBV-6VEIhmbOA9TMyCmKH35Q?e=hLAPNi).\n\n\n# Citation\nIf you find this project useful, please consider citing:\n\n```\n@inproceedings{lai2022stratified,\n  title={Stratified Transformer for 3D Point Cloud Segmentation},\n  author={Lai, Xin and Liu, Jianhui and Jiang, Li and Wang, Liwei and Zhao, Hengshuang and Liu, Shu and Qi, Xiaojuan and Jia, Jiaya},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={8500--8509},\n  year={2022}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdvlab-research%2FStratified-Transformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdvlab-research%2FStratified-Transformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdvlab-research%2FStratified-Transformer/lists"}