{"id":13443146,"url":"https://github.com/haibo-qiu/GFNet","last_synced_at":"2025-03-20T16:30:33.700Z","repository":{"id":43534275,"uuid":"507220341","full_name":"haibo-qiu/GFNet","owner":"haibo-qiu","description":"[TMLR 2022] Geometric Flow Network for 3D Point Cloud Semantic Segmentation","archived":false,"fork":false,"pushed_at":"2023-01-10T00:00:34.000Z","size":11331,"stargazers_count":38,"open_issues_count":1,"forks_count":7,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-28T06:57:35.821Z","etag":null,"topics":["deep-learning","point-cloud","pytorch","semantic-segmentation"],"latest_commit_sha":null,"homepage":"https://haibo-qiu.github.io/GFNet/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/haibo-qiu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-25T05:16:12.000Z","updated_at":"2024-05-17T18:19:02.000Z","dependencies_parsed_at":"2023-02-08T16:00:49.848Z","dependency_job_id":null,"html_url":"https://github.com/haibo-qiu/GFNet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haibo-qiu%2FGFNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haibo-qiu%2FGFNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haibo-qiu%2FGFNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haibo-qiu%2FGFNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/haibo-qiu","download_url":"https://codeload.github.com/haibo-qiu/GFNet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244649679,"owners_count":20487467,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","point-cloud","pytorch","semantic-segmentation"],"created_at":"2024-07-31T03:01:56.712Z","updated_at":"2025-03-20T16:30:31.204Z","avatar_url":"https://github.com/haibo-qiu.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# GFNet [![arXiv](https://img.shields.io/badge/arXiv-2207.02605-b31b1b)](https://arxiv.org/abs/2207.02605) [![TMLR](https://img.shields.io/badge/TMLR-2022.238-blue)](https://openreview.net/forum?id=LSAAlS7Yts) [![Project](https://img.shields.io/badge/Project-Page-important)](https://haibo-qiu.github.io/GFNet/)\nThis is the Pytorch implementation of our following paper:\n\u003e**[GFNet: Geometric Flow Network for 3D Point Cloud Semantic Segmentation](https://arxiv.org/abs/2207.02605)**\n\u003e\u003cbr\u003e*Accepted by [TMLR](https://openreview.net/forum?id=LSAAlS7Yts), 2022*\n\u003e\u003cbr\u003e*Haibo Qiu, Baosheng Yu and Dacheng Tao*\u003cbr\u003e\n\u003e\u003cdetails\u003e\u003csummary\u003e \u003cb\u003eAbstract\u003c/b\u003e\u003c/summary\u003e\n\u003e\n\u003ePoint cloud semantic segmentation from projected views, such as range-view (RV) and bird's-eye-view (BEV), has been intensively investigated. Different views capture different information of point clouds and thus are complementary to each other. However, recent projection-based methods for point cloud semantic segmentation usually utilize a vanilla late fusion strategy for the predictions of different views, failing to explore the complementary information from a geometric perspective during the representation learning. In this paper, we introduce a geometric flow network (GFNet) to explore the geometric correspondence between different views in an align-before-fuse manner. Specifically, we devise a novel geometric flow module (GFM) to bidirectionally align and propagate the complementary information across different views according to geometric relationships under the end-to-end learning scheme. We perform extensive experiments on two widely used benchmark datasets, SemanticKITTI and nuScenes, to demonstrate the effectiveness of our GFNet for project-based point cloud semantic segmentation. Concretely, GFNet not only significantly boosts the performance of each individual view but also achieves state-of-the-art results over all existing projection-based models. \n\u003e\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003e \u003cb\u003eSegmentation GIF\u003c/b\u003e\u003c/summary\u003e\n\n![vis](figs/vis.gif)\u003cbr\u003e\n(_A gif of segmentation results on [SemanticKITTI](http://semantic-kitti.org) by GFNet_)\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003e \u003cb\u003eFramework\u003c/b\u003e\u003c/summary\u003e\n\n![framework](figs/framework.png)\n![gfm](figs/gfm.png)\n\u003c/details\u003e\n\n**Table of Contents**\n* [Installation](#installation)\n* [Data preparation](#data-preparation)\n* [Training](#training)\n* [Inference](#inference)\n   * [SemanticKITTI](#semantickitti)\n   * [nuScenes](#nuscenes)\n* [Acknowledgment](#acknowledgment)\n* [Citation](#citation)\n\n## Installation\n1. Clone this repo:\n    ```bash\n    git clone https://github.com/haibo-qiu/GFNet.git\n    ```\n2. Create a conda env with\n   ```bash\n   conda env create -f environment.yml\n   ```\n   Note that we also provide the `Dockerfile` for an alternative setup method.\n\n## Data preparation\n\n1. Download point clouds data from [SemanticKITTI](http://semantic-kitti.org) and [nuScenes](https://www.nuscenes.org/nuscenes#download).\n2. For SemanticKITTI, directly unzip all data into `dataset/SemanticKITTI`.\n3. For nuScenes, first unzip data to `dataset/nuScenes/full` and then use the following cmd to generate pkl files for both training and testing:\n    ```bash\n    python dataset/utils_nuscenes/preprocess_nuScenes.py\n    ```\n4. Final data folder structure will look like:\n   ```\n      dataset\n      └── SemanticKITTI\n          └── sequences\n              ├── 00\n              ├── ...\n              └── 21\n      └── nuScenes\n          └── full\n              ├── lidarseg\n              ├── smaples\n              ├── v1.0-{mini, test, trainval}\n              └── ...\n          └── nuscenes_train.pkl\n          └── nuscenes_val.pkl\n          └── nuscenes_trainval.pkl\n          └── nuscenes_test.pkl\n\n    ```\n\n## Training\n- Please refer to `configs/semantic-kitti.yaml` and `configs/nuscenes.yaml` for dataset specific properties.\n- Download the [pretrained resnet model](https://drive.google.com/file/d/1I85xLRwUMIeW_7BvdZ4uZ0Lm4j3zxLT1/view?usp=sharing) to `pretrained/resnet34-333f7ec4.pth`.\n- The hyperparams for training are included in `configs/resnet_semantickitti.yaml` and `configs/resnet_nuscenes.yaml`. After modifying corresponding settings to satisfy your purpose, the network can be trained in an end-to-end manner by:\n    1. `./scripts/start.sh` on SemanticKITTI.\n    2. `./scripts/start_nuscenes.sh` on nuScenes.\n\n## Inference\n### SemanticKITTI\n1. Download [gfnet_63.0_semantickitti.pth.tar](https://drive.google.com/file/d/1J7jeSY5hGIHZO3WBdZnZLfdH-plv-81g/view?usp=sharing) into `pretrained/`.\n2. Evaluate on SemanticKITTI valid set by:\n    ```bash\n    ./scripts/infer.sh\n    ```\n    Alternatively, you can use the [official semantic-kitti api](https://github.com/PRBonn/semantic-kitti-api#evaluation) for evaluation.\n3. To reproduce the results we submitted to the test server:\n    1. download [gfnet_submit_semantickitti.pth.tar](https://drive.google.com/file/d/1bdq2_l5Q0tyww7wc3wlyrU09H2tVY5LF/view?usp=sharing) into `pretrained/`, \n    2. uncomment and run the second cmd in `./scripts/infer.sh`.\n    3. zip `path_to_results_folder/sequences` for submission.\n\n### nuScenes \n1. Download [gfnet_76.8_nuscenes.pth.tar](https://drive.google.com/file/d/1r5SXpToLBdiYdNp7we9Bw-Chmc0x7QPd/view?usp=sharing) into `pretrained/`.\n2. Evaluate on nuScenes valid set by:\n    ```bash\n    ./scripts/infer_nuscenes.sh\n    ```\n3. To reproduce the results we submitted to the test server:\n    1. download [gfnet_submit_nuscenes.pth.tar](https://drive.google.com/file/d/16nI5NjZ4wgNRwEC_HjqrVQLFLWXqThzs/view?usp=sharing) into `pretrained/`.\n    2. uncomment and run the second cmd in `./scripts/infer_nuscenes.sh`.\n    3. check the valid format of predictions by:\n        ```bash\n        ./dataset/utils_nuscenes/check.sh\n        ```\n        where `result_path` needs to be modified correspondingly.\n    4. submit the `dataset/nuScenes/preds.zip` to the test server.\n    \n## Acknowledgment\n\nThis repo is built based on [lidar-bonnetal](https://github.com/PRBonn/lidar-bonnetal), [PolarSeg](https://github.com/edwardzhou130/PolarSeg) and [kprnet](https://github.com/DeyvidKochanov-TomTom/kprnet). Thanks the contributors of these repos!\n\n## Citation\nIf you use our code or results in your research, please consider citing with:\n```bibtex\n@article{qiu2022gfnet,\n  title={{GFN}et: Geometric Flow Network for 3D Point Cloud Semantic Segmentation},\n  author={Haibo Qiu and Baosheng Yu and Dacheng Tao},\n  journal={Transactions on Machine Learning Research},\n  year={2022},\n  url={https://openreview.net/forum?id=LSAAlS7Yts},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaibo-qiu%2FGFNet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhaibo-qiu%2FGFNet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaibo-qiu%2FGFNet/lists"}