{"id":25657452,"url":"https://github.com/rscr1/pvo","last_synced_at":"2025-02-23T23:18:57.685Z","repository":{"id":235758095,"uuid":"791177838","full_name":"rscr1/PVO","owner":"rscr1","description":"code for \"PVO: Panoptic Visual Odometry\", CVPR 2023","archived":false,"fork":false,"pushed_at":"2024-04-30T19:56:19.000Z","size":44340,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-04-30T21:11:45.098Z","etag":null,"topics":["panoptic-segmentation","slam","visualodometry"],"latest_commit_sha":null,"homepage":"https://zju3dv.github.io/pvo/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rscr1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-24T08:30:39.000Z","updated_at":"2024-04-30T19:56:23.000Z","dependencies_parsed_at":"2024-04-30T21:13:04.380Z","dependency_job_id":null,"html_url":"https://github.com/rscr1/PVO","commit_stats":null,"previous_names":["rscr1/pvo"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rscr1%2FPVO","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rscr1%2FPVO/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rscr1%2FPVO/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rscr1%2FPVO/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rscr1","download_url":"https://codeload.github.com/rscr1/PVO/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240391057,"owners_count":19793834,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["panoptic-segmentation","slam","visualodometry"],"created_at":"2025-02-23T23:18:57.218Z","updated_at":"2025-02-23T23:18:57.665Z","avatar_url":"https://github.com/rscr1.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PVO: Panoptic Visual Odometry\n### [Project Page](https://zju3dv.github.io/pvo/) | [Paper](https://arxiv.org/abs/2207.01610)\n\u003cbr/\u003e\n\n\u003e PVO: Panoptic Visual Odometry  \n\n\u003e [[Weicai Ye](https://ywcmaike.github.io/), [Xinyue Lan](https://github.com/siyisan)]\u003csup\u003eCo-Authors\u003c/sup\u003e, [Shuo Chen](https://github.com/Eric3778), [Yuhang Ming](https://github.com/YuhangMing), [Xinyuan Yu](https://github.com/RickyYXY), [Hujun Bao](http://www.cad.zju.edu.cn/home/bao/), [Zhaopeng Cui](https://zhpcui.github.io/), [Guofeng Zhang](http://www.cad.zju.edu.cn/home/gfzhang)\n\n\u003e CVPR 2023\n\n![demo_vid](assets/pvo_teaser.gif)\n\n## Test on vkitti 15-deg-left datasets.\n0) prepare.\nfollow [prepare.md](prepare.md)\n\n1) generate inital panoptic segmentation.\n```\nsh tools/initial_segmentation.sh  \n```\n\n2) vps-\u003evo，vo Module generate pose, flow and depth.\n```\nsh tools/test_vo_scene.sh  \n```\n\n3) vo-\u003evps, vps Module use flow and depth from vo Module and generate final video panoptic segmentation results and vpq.\n```\nsh tools/test_vps.sh  \n```\n\n## Metrics on Virtual_KITTI2\n|Scene|RMSE|vpq_all/vpq_thing/vpq_stuff|\n|-----|----|---------------------------|\n|Scene01|0.371|40.39/26.43/44.57|\n|Scene02|0.058|68.84/88.83/62.18|\n|Scene06|0.113|66.38/79.99/62.97|\n|Scene18|0.951|68.35/83.86/63.92|\n|Scene20|3.503|35.11/16.83/40.59|\n\nYou can get the results in the paper by iterating multiple times.\n\n## Train on vkitti 15-deg-left datasets.\n1)  To train VPS_Module, you can refer to [Detectron2](https://detectron2.readthedocs.io/en/latest/tutorials/getting_started.html) for more training details.\nHere for example, you can train  vkitti 15-deg-left on 4 GPUs, and training results are saved on `output/vps_training/`. You can modify the config-file according to the hardware conditions.\n```\npython -W ignore VPS_Module/tools/train_net.py \\\n--config-file VPS_Module/configs/COCO-PanopticSegmentation/panoptic_fpn_R_50_3x_vkitti_511.yaml --num-gpu 4 \\\nMODEL.WEIGHTS checkpoints/panFPN.pth \\\nOUTPUT_DIR output/vps_training/\n```\n\nAnd you can use tools/train_net.sh script instead that command\n\n2) To train VO_Module, you can refer to [DROID-SLAM](https://github.com/princeton-vl/DROID-SLAM) for more training details.\nHere for example, you can train vkitti on 4 GPUs.\n```\npython VO_Module/train.py --gpus=4 --lr=0.00025\n```\n\n## Visualization\nYou can refer to [DROID-SLAM](https://github.com/princeton-vl/DROID-SLAM) for visualization.\nAll demos can be run on a GPU with 11G of memory. While running, press the \"s\" key to increase the filtering threshold (= more points) and \"a\" to decrease the filtering threshold (= fewer points).\n```\npython VO_Module/evaluation_scripts/test_vo.py --datapath=datasets/Virtual_KITTI2/Scene01 --segm_filter True \n```\n\n## Citation\n\nIf you find this code useful for your research, please use the following BibTeX entry.\n\n```bibtex\n\n@inproceedings{Ye2023PVO,\n  title={{PVO: Panoptic visual odometry}},\n  author={Ye, Weicai and Lan, Xinyue and Chen, Shuo and Ming, Yuhang and Yu, Xingyuan and Bao, Hujun and Cui, Zhaopeng and Zhang, Guofeng},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={9579--9589},\n  year={2023}\n}\n\n```\n\n## Acknowledgement\n\nSome code snippets are borrowed from [DROID-SLAM](https://github.com/princeton-vl/DROID-SLAM) and [Detectron2](https://github.com/facebookresearch/detectron2). Thanks for these great projects.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frscr1%2Fpvo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frscr1%2Fpvo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frscr1%2Fpvo/lists"}