{"id":15480993,"url":"https://github.com/nubot-nudt/InsMOS","last_synced_at":"2025-10-12T01:31:07.124Z","repository":{"id":102291496,"uuid":"606353251","full_name":"nubot-nudt/InsMOS","owner":"nubot-nudt","description":"[IROS 23] InsMOS: Instance-Aware Moving Object Segmentation in LiDAR Data","archived":false,"fork":false,"pushed_at":"2025-06-25T01:43:20.000Z","size":19225,"stargazers_count":139,"open_issues_count":0,"forks_count":6,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-06-25T02:39:55.777Z","etag":null,"topics":["lidar","lidar-segmentation","motion-detection","motion-segmentation","moving-object-segmentation","point-cloud-segmentation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nubot-nudt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-02-25T08:19:51.000Z","updated_at":"2025-06-25T01:43:23.000Z","dependencies_parsed_at":"2025-06-25T02:31:01.991Z","dependency_job_id":"aed29d20-bd44-45e4-91cb-fc010ff10fba","html_url":"https://github.com/nubot-nudt/InsMOS","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nubot-nudt/InsMOS","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nubot-nudt%2FInsMOS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nubot-nudt%2FInsMOS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nubot-nudt%2FInsMOS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nubot-nudt%2FInsMOS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nubot-nudt","download_url":"https://codeload.github.com/nubot-nudt/InsMOS/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nubot-nudt%2FInsMOS/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279009769,"owners_count":26084648,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lidar","lidar-segmentation","motion-detection","motion-segmentation","moving-object-segmentation","point-cloud-segmentation"],"created_at":"2024-10-02T05:00:59.932Z","updated_at":"2025-10-12T01:31:07.119Z","avatar_url":"https://github.com/nubot-nudt.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# InsMOS: Instance-Aware Moving Object Segmentation in LiDAR Data\n\n[![](https://img.shields.io/badge/Page-InsMOS-d6ccc2?style=flat-square\u0026labelColor=495057\u0026logoColor=white)](https://neng-wang.github.io/InsMOS/)  [![](https://img.shields.io/badge/Paper-IEEE-023e8a?style=flat-square\u0026labelColor=495057\u0026logoColor=white)](https://ieeexplore.ieee.org/document/10342277)  [![](https://img.shields.io/badge/Paper-arXiv-d62828?style=flat-square\u0026labelColor=495057\u0026logoColor=white)](https://arxiv.org/abs/2303.03909)\n\n\u003c/div\u003e\n\n#### **We propose an extended version, named [SegNet4D](https://github.com/nubot-nudt/SegNet4D), featuring  complete 4D semantic segmentation capability and enhanced motion  recognition performance.**\n\nThis repository contains the implementation of our paper:\n\n\u003e **InsMOS: Instance-Aware Moving Object Segmentation in LiDAR Data**\\\n\u003e [Neng Wang](https://github.com/neng-wang),  [Chenghao Shi](https://github.com/chenghao-shi),  Ruibin Guo,  Huimin Lu,  Zhiqiang Zheng,  [Xieyuanli Chen](https://github.com/Chen-Xieyuanli)   \n\n\u003cdiv align=center\u003e\n\u003cimg src=\"./docs/InsMOS.gif\"\u003e \n\u003c/div\u003e\n\n- *Our instance-aware moving object segmentation on the SemanticKITTI sequence 08 and 20, 21.*\n\n- *Red points indicate predicted moving points, cyan indicate predicted static instance points and gray points are static background.*\n\n- *Green bounding boxes represent cars, blue bounding boxes represent pedestrians, and yellow bounding boxes represent cyclists.*\n\n## News\n\n- [2023-8-12] Code released!\n- [2023-6-22] Our work is accepted for IROS2023 :clap:\n\n  \n## Citation\n\nIf you use our code in your work, please star our repo and cite our paper.\n\n```bibtex\n@inproceedings{wang2023iros,\n\ttitle={{InsMOS: Instance-Aware Moving Object Segmentation in LiDAR Data}},\n\tauthor={Wang, Neng and Shi, Chenghao and Guo, Ruibin and Lu, Huimin and Zheng, Zhiqiang and Chen, Xieyuanli},\n\tbooktitle={IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},\n        pages={7598-7605},\n\tyear={2023}\n}\n```\n\n## Data\n\n1、SemanticKITTI: Download SemanticKITTI dataset from the official [website](http://semantic-kitti.org/). \n\n2、KITTI-road Dataset: Download the KITTI-road Velodyne point clouds from the official [website](https://www.cvlibs.net/datasets/kitti/raw_data.php?type=road) and MOS label from [MotionSeg3D](https://github.com/haomo-ai/MotionSeg3D).\n\n3、Instance label:   Download  the box labels from [ondrive](https://1drv.ms/f/s!Ak6KrcxOqwZfkABaeJYYLb7ZT7Fg?e=zguXiK) or [BaiduDisk,code:59t7](https://pan.baidu.com/s/1TVBED6KZmEsJI6R_xjdLRQ?pwd=59t7), and please refer to [boundingbox_label_readme](./dataloader/boundingbox_label_readme.md) about more details of instance label .\n\n\u003cdetails\u003e\n    \u003csummary\u003e\u003cstrong\u003eData structure\u003c/strong\u003e\u003c/summary\u003e\n\n```\n└── sequences\n  ├── 00/           \n  │   ├── velodyne/\t\n  |   |\t├── 000000.bin\n  |   |\t├── 000001.bin\n  |   |\t└── ...\n  │   ├── labels/ \n  |   | ├── 000000.label\n  |   | ├── 000001.label\n  |   | └── ...\n  |   ├── boundingbox_label\n  |   | ├── 000000.npy\n  |   |\t├── 000001.npy\n  |   |\t└── ...\n  |   ├── calib.txt\n  |   ├── poses.txt\n  |   └── times.txt\n  ├── 01/ # 00-10 for training\n  ├── 08/ # for validation\n  ├── 11-21/ # 11-21 for testing\n  # kitti-road\n  ├── 30 31 32 33 34 40 # for training\n  └── 35 36 37 38 39 41 # for testing\n```\n\n\u003c/details\u003e \n\n## Installation\n\n#### 1. Dependencies\n\n **system dependencies:**\n\n```bash\nubuntu20.04, CUDA 11.3, cuDNN 8.2.1, \n```\n\n**python dependencies:**\n\n```bash\npython 3.7\n```\n\n#### 2. Set up conda environment\n\n```bash\nconda create --name insmos python=3.7\nconda activate insmos\npip install -r requirements.txt\n\n# insltall pytorch with cuda11.3, avoid using \"pip install torch\"\nconda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=11.3 -c pytorch -c conda-forge\n\n# ensure numpy==1.18.1\npip uninstall numpy\npip install numpy==1.18.1\n```\n\n**Install MinkowskiEngine :**\n\n```bash\ncd ~\nmkdir ThirdParty\nsudo apt-get install libopenblas-dev\ngit clone https://github.com/NVIDIA/MinkowskiEngine.git\ncd MinkowskiEngine\nconda activate insmos\npython setup.py install --blas_include_dirs=${CONDA_PREFIX}/include --blas=openblas\n```\n\n#### 3. Install InsMOS\n\n```bash\n# clone code\ngit clone https://github.com/nubot-nudt/InsMOS.git\ncd InsMOS\n\n# activate conda\nconda activate insmos\n\n# install\npython setup.py develop\n```\n\n## Inference\n\nRun the following command to evaluate the model in SemanticKITTI validation dataset or test dataset. At this moment, a “preb_out” folder will be generated, which contains  “bbox_preb”, “confidence”, and “mos_preb” for storing the predicted  bounding boxes, confidence scores for moving points, and labels for  moving points, respectively.\n\nWe public the model was trained on the SemanticKITTI dataset (N_10_t_0.1_odom.ckpt) and the other model was trained on the Semantic-KITTI and KITTI-road dataset (N_10_t_0.1_odom_road.ckpt). You can download from  [ondrive](https://1drv.ms/f/s!Ak6KrcxOqwZfkABaeJYYLb7ZT7Fg?e=zguXiK) or [BaiduDisk,code:59t7](https://pan.baidu.com/s/1TVBED6KZmEsJI6R_xjdLRQ?pwd=59t7), and then put the model in \"ckpt\" folder.\n\n```bash\ncd InsMOS\npython scripts/predict_mos.py --cfg_file config/config.yaml --data_path /path/to/kitti/sequences --ckpt ./ckpt/N_10_t_0.1_odom.ckpt --split valid\n```\n\n### Evaluate\n\nWe use the [semantickitti-api](https://github.com/PRBonn/semantic-kitti-api) to evaluate the MOS IOU.\n\n```bash\ncd semantic-kitti-api\npython evaluate_mos.py --dataset /path/to/kitti --predictions ./preb_out/InsMOS/mos_preb --split valid\n```\n\n### Refine\n\nRun the following command to refine the network predictions.\n\n```bash\npython scripts/refine.py --data_path /path/to/kitti/sequences --split valid\n```\n\n### Re-evaluate the refinement\n\nRe-evaluate the results of refinement.\n\n```bash\npython evaluate_mos.py --dataset /path/to/kitti --predictions ./preb_out_refine/mos_preb --split valid \n```\n\n### Visual\n\nRun the following command to visualize the results of moving object segmentation and instance prediction.\n\nPress key  `n`  to show next frame.\n\nPress key  `b`  to show last frame.\n\nPress key  `q`  to quit display.\n\n```bash\ncd visual\npython vis_mos_bbox.py\n```\n\n### Train\n\nYou can set  single gpu or multi gpu for training  in [train.py](./scripts/train.py). We set batch size to 4 for each gpu. During the training process, there may be an increase in GPU memory  consumption, so it is advisable not to set the batch size too large  initially. We test 4-6 is fine on  3090 GPU. \n\n```bash\nexport DATA=/path/to/kitti/sequences\npython scripts/train.py\n```\n\nIf the training process is interrupted unexpectedly, you can resume the training using the following command. \n\n```bash\npython scripts/train.py --weights ./logs/InsMOS/version_x/checkpoints/last.ckpt --checkpoint ./logs/InsMOS/version_x/checkpoints/last.ckpt\n```\n\n## Contact\n\nAny question or suggestions are welcome!\n\nNeng Wang: nwang@nudt.edu.cn and Xieyuanli Chen: [website](https://github.com/Chen-Xieyuanli)\n\n## License\n\nThis project is free software made available under the MIT License. For details see the LICENSE file.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnubot-nudt%2FInsMOS","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnubot-nudt%2FInsMOS","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnubot-nudt%2FInsMOS/lists"}