{"id":19217014,"url":"https://github.com/hustvl/osp","last_synced_at":"2025-09-04T06:40:24.294Z","repository":{"id":247323527,"uuid":"730112598","full_name":"hustvl/osp","owner":"hustvl","description":"[ECCV 2024] Occupancy as Set of Points","archived":false,"fork":false,"pushed_at":"2024-07-08T02:42:50.000Z","size":35523,"stargazers_count":88,"open_issues_count":0,"forks_count":4,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-28T23:34:38.091Z","etag":null,"topics":["3d","3d-occupancy-prediction","autonomous-driving","occupancy"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hustvl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-11T08:30:11.000Z","updated_at":"2025-03-22T10:08:45.000Z","dependencies_parsed_at":"2024-07-08T04:59:43.662Z","dependency_job_id":null,"html_url":"https://github.com/hustvl/osp","commit_stats":null,"previous_names":["hustvl/osp"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hustvl%2Fosp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hustvl%2Fosp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hustvl%2Fosp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hustvl%2Fosp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hustvl","download_url":"https://codeload.github.com/hustvl/osp/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249129209,"owners_count":21217304,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","3d-occupancy-prediction","autonomous-driving","occupancy"],"created_at":"2024-11-09T14:19:52.151Z","updated_at":"2025-04-15T18:32:48.108Z","avatar_url":"https://github.com/hustvl.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\u003ch1\u003eOccupancy as Set of Points \u003c/h1\u003e\n\n[Yiang Shi](https://github.com/shawnsya)\u003csup\u003e1,\\*\u003c/sup\u003e, [Tianheng Cheng](https://scholar.google.com/citations?user=PH8rJHYAAAAJ)\u003csup\u003e1,\\*\u003c/sup\u003e, [Qian Zhang](https://scholar.google.com/citations?user=pCY-bikAAAAJ\u0026hl=zh-CN)\u003csup\u003e2\u003c/sup\u003e, [Wenyu Liu](http://eic.hust.edu.cn/professor/liuwenyu/)\u003csup\u003e1\u003c/sup\u003e, [Xinggang Wang](https://xwcv.github.io/)\u003csup\u003e1 :email:\u003c/sup\u003e\n \n\u003csup\u003e1\u003c/sup\u003e  [School of EIC, HUST](http://english.eic.hust.edu.cn/),\n\u003csup\u003e2\u003c/sup\u003e  [Horizon Robotics](https://en.horizonrobotics.com/)\n\n\n\\* equal contribution, \u003csup\u003e:email:\u003c/sup\u003e corresponding author.\n\n[![arxiv paper](https://img.shields.io/badge/arXiv-Paper-red)](https://arxiv.org/abs/2407.04049)\n\n\n**ECCV 2024**\n\n\u003c/div\u003e\n\n\n#\n\n### News\n* `[2024-7-8]` We have released the [arXiv paper](https://arxiv.org/abs/2407.04049) of OSP.\n* `[2024-7-2]` OSP is accepted by ECCV 2024!\n\n\n## Abstract\nIn this paper, we explore a novel point representation for 3D occupancy prediction from multi-view images, which is named Occupancy as Set of Points. Existing camera-based methods tend to exploit dense volume-based representation to predict the occupancy of the whole scene, making it hard to focus on the special areas or areas out of the perception range. In comparison, we present the \\textit{Points of Interest} (PoIs) to represent the scene and propose OSP, a novel framework for point-based 3D occupancy prediction. Owing to the inherent flexibility of the point-based representation, OSP achieves strong performance compared with existing methods and excels in terms of training and inference adaptability. It extends beyond traditional perception boundaries and can be seamlessly integrated with volume-based methods to significantly enhance their effectiveness. Experiments on the Occ3D-nuScenes occupancy benchmark show that OSP has strong performance and flexibility.\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"assets/architecture.png\" /\u003e\n\u003c/div\u003e\n\n## Preliminary\n\n### Installation\n1. Prepare conda environment referring to the documentation of [BEVFormer](https://github.com/fundamentalvision/BEVFormer/blob/master/docs/install.md)\n\n### Prepare Dataset\n1. Download nuScenes and prepare annotations referring to the documentation of [3D Occupancy Prediction Challenge at CVPR 2023](https://github.com/CVPR2023-3D-Occupancy-Prediction/CVPR2023-3D-Occupancy-Prediction)\n\n### Pretrained Weights \nThe pretrained weight of fcos3d can be downloaded [here](https://github.com/zhiqi-li/storage/releases/download/v1.0/r101_dcn_fcos3d_pretrain.pth)\n\n\n## Usage \n1. **Training**\n    ```shell\n    bash train.sh\n    ```\n    * Replace the default config file as needed.\n    * Config osp_minibatch.py represents mini dataset of nuScenes. \n\n\n2. **Evaluation**\n    ```shell\n    bash test.sh\n    ```\n    * Replace the default config file as needed.\n    * Replace the checkpoint path in the script with your own.\n\n## Results\n| Backbone | Method | Lr Schd | IoU|  Config | Download |\n| :---: | :---: | :---: | :---: | :---: | :---: |\n| R101 | OSP | 24ep | 39.41 |[config](projects/configs/osp/osp.py) |[model]()|\n| R101 | BEVFormer w/ OSP | 24ep | 41.21 |[config](projects/configs/osp/osp.py) |[model]()|\n\n* Model weights will be released later.\n\n## Citations\n```bibtex\n@inproceedings{shi2024occupancysetpoints,\n      title={Occupancy as Set of Points}, \n      author={Yiang Shi and Tianheng Cheng and Qian Zhang and Wenyu Liu and Xinggang Wang},\n      year={2024},\n      booktitle={ECCV}\n}\n```\n\n\n## License\n\nReleased under the [MIT](LICENSE) License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhustvl%2Fosp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhustvl%2Fosp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhustvl%2Fosp/lists"}