{"id":13441141,"url":"https://github.com/I2-Multimedia-Lab/Pointsoup","last_synced_at":"2025-03-20T11:35:31.351Z","repository":{"id":234862523,"uuid":"787730550","full_name":"I2-Multimedia-Lab/Pointsoup","owner":"I2-Multimedia-Lab","description":"[IJCAI 2024] Pointsoup: High-Performance and Extremely Low-Decoding-Latency Learned Geometry Codec for Large-Scale Point Cloud Scenes","archived":false,"fork":false,"pushed_at":"2024-04-23T01:38:00.000Z","size":14634,"stargazers_count":12,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-08-01T03:33:19.078Z","etag":null,"topics":["3d","compression","deep-learning","point-cloud"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2404.13550","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/I2-Multimedia-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-04-17T04:29:36.000Z","updated_at":"2024-05-27T07:47:21.000Z","dependencies_parsed_at":"2024-04-21T07:38:10.257Z","dependency_job_id":null,"html_url":"https://github.com/I2-Multimedia-Lab/Pointsoup","commit_stats":null,"previous_names":["i2-multimedia-lab/pointsoup"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/I2-Multimedia-Lab%2FPointsoup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/I2-Multimedia-Lab%2FPointsoup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/I2-Multimedia-Lab%2FPointsoup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/I2-Multimedia-Lab%2FPointsoup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/I2-Multimedia-Lab","download_url":"https://codeload.github.com/I2-Multimedia-Lab/Pointsoup/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221752311,"owners_count":16874958,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","compression","deep-learning","point-cloud"],"created_at":"2024-07-31T03:01:30.394Z","updated_at":"2025-03-20T11:35:31.333Z","avatar_url":"https://github.com/I2-Multimedia-Lab.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Pointsoup: High-Performance and Extremely Low-Decoding-Latency Learned Geometry Codec for Large-Scale Point Cloud Scenes\n\n## News\n- [2024.4.23] The manuscript is now available on Arxiv [2404.13550](https://arxiv.org/abs/2404.13550).\n- [2024.4.21] The supplementary material is uploaded to [Google Drive](https://drive.google.com/file/d/113PvrVBll9frY1k6OC3QDA5PugnDcHdV/view?usp=sharing).\n- [2024.4.17] Our paper has been accepted by [IJCAI 2024](https://doi.org/10.24963/ijcai.2024/595)!\n\n## Overview\n\u003e Despite considerable progress being achieved in point cloud geometry compression, there still remains a challenge in effectively compressing large-scale scenes with sparse surfaces. Another key challenge lies in reducing decoding latency, a crucial requirement in real-world application. In this paper, we propose Pointsoup, an efficient learning-based geometry codec that attains high-performance and extremely low-decoding-latency simultaneously. Inspired by conventional Trisoup codec, a point model-based strategy is devised to characterize local surfaces. Specifically, skin features are embedded from local windows via an attention-based encoder, and dilated windows are introduced as cross-scale priors to infer the distribution of quantized features in parallel. During decoding, features undergo fast refinement, followed by a folding-based point generator that reconstructs point coordinates with fairly fast speed. Experiments show that Pointsoup achieves state-of-the-art performance on multiple benchmarks with significantly lower decoding complexity, i.e., up to 90$\\sim$160$\\times$ faster than the G-PCCv23 Trisoup decoder on a comparatively low-end platform (e.g., one RTX 2080Ti). Furthermore, it offers variable-rate control with a single neural model (2.9MB), which is attractive for industrial practitioners.\n\n## Environment\n\nThe environment we use is as follows：\n\n- Python 3.10.14\n- Pytorch 2.0.1 with CUDA 11.7\n- Pytorch3d 0.7.5\n- Torchac 0.9.3\n\nFor the convenience of reproduction, we provide three different ways to help create the environment:\n\n#### Option 1: Using yml\n\n```\nconda env create -f=./environment/environment.yml\n```\n\n#### Option 2: Using .sh\n\n```\nsource ./environment/env_create.sh\n```\n\n#### Option 3: CodeWithGPU (AutoDL image)\n\n[Pointsoup Image](https://www.codewithgpu.com/i/I2-Multimedia-Lab/Pointsoup/Pointsoup) has been uploaded at [CodeWithGPU](https://www.codewithgpu.com/image) community. The required environment can be instantly built once you create an [AutoDL](https://www.autodl.com) container instance with our image `I2-Multimedia-Lab/Pointsoup/Pointsoup` being selected from the community image list.\n\n## Data\n\nIn our paper, point clouds with the coordinate range of [0, 1023] are used as input.\n\nExample point clouds are saved in ``./data/example_pc_1023/``, trained model is saved in ``./model/exp/``.\n\n## Compression\nFirst and foremost, the `tmc3` is need to perform predtree coding on bone points. If the `tmc3` file we provided cannot work on your platform, please refer to [MPEGGroup/mpeg-pcc-tmc13](https://github.com/MPEGGroup/mpeg-pcc-tmc13) for manual building.\n\n```\nchmod +x ./tmc3\n```\n\nYou can adjust the compression ratio by simply adjusting the parameter `local_window_size`. In our paper, we use `local_window_size` in the range of 2048~128.\n\n```\npython ./compress.py \\\n    --input_glob='./data/example_pc_1023/*.ply' \\\n    --compressed_path='./data/compressed/' \\\n    --model_load_path='./model/exp/ckpt.pt'\\\n    --local_window_size=200 \\\n    --tmc_path='./tmc3'\\\n    --verbose=True\n```\n\n## Decompression\n\n```\npython ./decompress.py \\\n    --compressed_path='./data/compressed/' \\\n    --decompressed_path='./data/decompressed/' \\\n    --model_load_path='./model/exp/ckpt.pt'\\\n    --tmc_path='./tmc3'\\\n    --verbose=True\n```\n\n## Evaluation\n\nWe use `PccAppMetrics` for D1 PSNR calculation. You can refer to [MPEGGroup/mpeg-pcc-tmc2](https://github.com/MPEGGroup/mpeg-pcc-tmc2) if the provided `PccAppMetrics` file does not fit your platform.\n\n```\nchmod +x ./PccAppMetrics\n```\n\n```\npython ./eval_PSNR.py \\\n    --input_glob='./data/example_pc_1023/*.ply' \\\n    --decompressed_path='./data/decompressed/' \\\n    --pcc_metric_path='./PccAppMetrics' \\\n    --resolution=1023\n```\n\n## Disucssion\nMerits:\n\n- High Performance - SOTA efficiency on multiple large-scale benchmarks.\n- Low Decoding Latency - 90~160× faster than the conventional Trisoup decoder.\n- Robust Generalizability - Applicable to large-scale samples once trained on small objects.\n- High Flexibility - Variable-rate control with a single neural model.\n- Light Weight - Fairly small with 761k parameters (about 2.9MB).\n\nLimitations:\n\n- Rate-distortion performance is inferior to G-PCC Octree codec at high bitrates (e.g., bpp\u003e1). The surface approximation-based approaches (Pointsoup and Trisoup) seem hard to characterize accurate point positions even if given enough bitrate budget.\n\n- Naive outdoor LiDAR frame coding efficacy is unsatisfactory. Due to the used sampling\u0026grouping strategy, the pointsoup is limited to point clouds with relatively uniform distributed points, such as [S3DIS](http://buildingparser.stanford.edu/dataset.html), [ScanNet](https://github.com/ScanNet/ScanNet), [dense point cloud map](https://github.com/PRBonn/deep-point-map-compression), [8iVFB (human body)](https://plenodb.jpeg.org/pc/8ilabs), [Visionair (objects)](https://github.com/yulequan/PU-Net), etc.\n\n\n## Citation\n\nIf you find this work useful, please consider citing our work:\n\n```\n@inproceedings{ijcai2024p595,\n  title     = {Pointsoup: High-Performance and Extremely Low-Decoding-Latency Learned Geometry Codec for Large-Scale Point Cloud Scenes},\n  author    = {You, Kang and Liu, Kai and Yu, Li and Gao, Pan and Ding, Dandan},\n  booktitle = {Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, {IJCAI-24}},\n  publisher = {International Joint Conferences on Artificial Intelligence Organization},\n  editor    = {Kate Larson},\n  pages     = {5380--5388},\n  year      = {2024},\n  month     = {8},\n  note      = {Main Track},\n  doi       = {10.24963/ijcai.2024/595},\n  url       = {https://doi.org/10.24963/ijcai.2024/595},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FI2-Multimedia-Lab%2FPointsoup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FI2-Multimedia-Lab%2FPointsoup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FI2-Multimedia-Lab%2FPointsoup/lists"}