{"id":18322507,"url":"https://github.com/tencentarc/sgat4pass","last_synced_at":"2025-04-05T23:31:04.553Z","repository":{"id":168467719,"uuid":"641775892","full_name":"TencentARC/SGAT4PASS","owner":"TencentARC","description":"This is the official implementation of the paper SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation (IJCAI 2023)","archived":false,"fork":false,"pushed_at":"2023-06-20T06:59:43.000Z","size":5658,"stargazers_count":30,"open_issues_count":0,"forks_count":3,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-03-21T13:23:08.690Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TencentARC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-17T06:32:17.000Z","updated_at":"2025-03-04T06:37:03.000Z","dependencies_parsed_at":null,"dependency_job_id":"49efc2c9-c146-42f6-ab61-bac567fcf811","html_url":"https://github.com/TencentARC/SGAT4PASS","commit_stats":null,"previous_names":["tencentarc/sgat4pass"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FSGAT4PASS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FSGAT4PASS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FSGAT4PASS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FSGAT4PASS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TencentARC","download_url":"https://codeload.github.com/TencentARC/SGAT4PASS/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247415783,"owners_count":20935383,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T18:24:54.059Z","updated_at":"2025-04-05T23:31:04.547Z","avatar_url":"https://github.com/TencentARC.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SGAT4PASS\n\n## Introduction\nThis is the official implementation of the paper SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation (IJCAI 2023)[[**PDF**](https://arxiv.org/pdf/2306.03403.pdf)].\n![SGAT4PASS](figs/pipeline.png)\n\n## Environments\n\n```bash\nconda create -n SGAT4PASS python=3.7.7\nconda activate SGAT4PASS\ncd ~/path/to/SGAT4PASS \nconda install pytorch==1.8.1 torchvision==0.9.1 torchaudio==0.8.1 cudatoolkit=11.1 -c pytorch -c conda-forge\npip install -r requirements.txt\npython setup.py develop --user\n# Optional: install apex follow: https://github.com/NVIDIA/apex\n```\n\n## Data Preparation\n\nPrepare datasets: \n\n- [Stanford2D3D](http://buildingparser.stanford.edu/dataset.html)\n\n```\ndatasets/\n├── Stanford2D3D\n│   ├── area_1\n│   ├── area_2\n│   ├── area_3\n│   ├── area_4\n│   ├── area_5a\n│   ├── area_5b\n│   └── area_6\n```\nPrepare pretrained weights, which can be found in the public repository of [SegFormer](https://github.com/NVlabs/SegFormer).\n```\npretrained/\n├── mit_b1.pth\n└── mit_b2.pth\n```\n## Train\n\nFor example, to use 4 A100 GPUs to run the experiments on Stanford2D3D dataset:\n\n```bash\n\nLR=0.00008\nEPOCHS=150\nPER_PIXEL_WEIGHT=0.3 # Panorama-Aware Loss Weight\nOFFSET_WEIGHT=0.3 # SDPE Loss Weight\nREPROJECTION=True # whether to use reprojection\nX_MAX=10 # the maximum value of x-axis\nY_MAX=10 # the maximum value of y-axis\nZ_MAX=360 # the maximum value of z-axis\nFOLD=1 # the fold of the Stanford2D3D dataset\n\npython -m torch.distributed.launch --nproc_per_node=4 --master_port 30005 tools/train_s2d3d_span.py \\\n    --config-file configs/stanford2d3d_pan/SGAT4PASS_small_1080x1080_line_axis_a100_xyz_mask_loss.yaml \\\n    TRAIN.MODEL_SAVE_DIR workdirs/SGAT4PASS_small_1080x1080_line_${OFFSET_WEIGHT}_axis_${OFFSET_WEIGHT}_mask_loss_${PER_PIXEL_WEIGHT}_lr${LR}_epoch_${EPOCHS} \\\n    SOLVER.LR $LR \\\n    TRAIN.EPOCHS $EPOCHS \\\n    SOLVER.PER_PIXEL_WEIGHT $PER_PIXEL_WEIGHT \\\n    SOLVER.OFFSET_LINE_WEIGHT $OFFSET_WEIGHT \\\n    SOLVER.OFFSET_AXIS_WEIGHT $OFFSET_WEIGHT \\\n    AUG.REPROJECTION $REPROJECTION \\\n    DATASET.FOLD $FOLD \\\n    AUG.X_MAX $X_MAX \\\n    AUG.Y_MAX $Y_MAX \\\n    AUG.Y_MAX $Z_MAX\n\n```\n\n## Test\nDownload the models from [GoogleDrive](https://drive.google.com/file/d/11MrFL6bThXFGIZEr_GE0HCivfBbZ3Dnv/view?usp=sharing) and put them in `./checkpoints/` folder.\n\n```\n./checkpoints\n├── Stanford2D3D_Fold_1\n│   ├── best_model_fold_1.pth\n├── Stanford2D3D_Fold_2\n│   ├── best_model_fold_2.pth\n└── Stanford2D3D_Fold_3\n    ├── best_model_fold_3.pth\n```\n\n### Traditional Testing\n\n```bash\npython -m torch.distributed.launch --nproc_per_node=4 --master_port 30005 tools/eval_s2d3d_span.py \\\n    --config-file configs/stanford2d3d_pan/SGAT4PASS_small_1080x1080_Fold_1.yaml \\\n    TEST.TEST_MODEL_PATH checkpoints/Stanford2D3D_Fold_3/best_model_fold_1.pth \\\n    DATASET.FOLD 1\n\npython -m torch.distributed.launch --nproc_per_node=4 --master_port 30005 tools/eval_s2d3d_span.py \\\n    --config-file configs/stanford2d3d_pan/SGAT4PASS_small_1080x1080_Fold_2.yaml \\\n    TEST.TEST_MODEL_PATH checkpoints/Stanford2D3D_Fold_3/best_model_fold_2.pth \\\n    DATASET.FOLD 2\n\npython -m torch.distributed.launch --nproc_per_node=4 --master_port 30005 tools/eval_s2d3d_span.py \\\n    --config-file configs/stanford2d3d_pan/SGAT4PASS_small_1080x1080_Fold_3.yaml \\\n    TEST.TEST_MODEL_PATH checkpoints/Stanford2D3D_Fold_3/best_model_fold_3.pth \\\n    DATASET.FOLD 3\n\n```\n\n### SGA Testing\nFirst, change the value of the `cfg.TEST.ROTATIONS` to the angle to be tested in the `segmentron/config/settings.py`.\n\nThen, run bash as follows\n```bash\n\npython -m torch.distributed.launch --nproc_per_node=4 --master_port 30005 tools/eval_s2d3d_span.py \\\n    --config-file configs/stanford2d3d_pan/SGAT4PASS_small_1080x1080_Fold_1.yaml \\\n    TEST.TEST_MODEL_PATH checkpoints/Stanford2D3D_Fold_1/best_model_fold_1.pth \\\n    TEST.SGA True \\\n    DATASET.FOLD 1\n\n```\nNote that `TEST.SGA` must be set to True\n\n## References\nOur repository is heavily based on [Trans4PASS](https://github.com/jamycheung/Trans4PASS)\n\nThank them for their excellent work!\n\n## License\n\nThis repository is under the Apache-2.0 license. For commercial use, please contact with the authors.\n\n\n## Citations\n\nIf you are interested in this work, please cite the following works:\n\nSGAT4PASS [[**PDF**](https://arxiv.org/pdf/2306.03403.pdf)]\n```\n@article{li2023sgat4pass,\n  title={SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation},\n  author={Li, Xuewei and Wu, Tao and Qi, Zhongang and Wang, Gaoang and Shan, Ying and Li, Xi},\n  journal={arXiv preprint arXiv:2306.03403},\n  year={2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftencentarc%2Fsgat4pass","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftencentarc%2Fsgat4pass","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftencentarc%2Fsgat4pass/lists"}