{"id":23515285,"url":"https://github.com/heethesh/salsa-semantic-assisted-slam","last_synced_at":"2025-06-21T04:38:05.265Z","repository":{"id":98283151,"uuid":"258530194","full_name":"heethesh/SALSA-Semantic-Assisted-SLAM","owner":"heethesh","description":"SALSA: Semantic Assisted Life-Long SLAM for Indoor Environments (16-833 SLAM Project at CMU)","archived":false,"fork":false,"pushed_at":"2020-07-09T21:18:19.000Z","size":80744,"stargazers_count":76,"open_issues_count":0,"forks_count":12,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-11T14:32:59.129Z","etag":null,"topics":["indoor-environments","lifelong-slam","loop-closure","mask-rcnn","openloris-scene","orb-slam2","salsa","semantic-slam","slam","slam-methods","tum-rgbd"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/heethesh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-24T14:09:07.000Z","updated_at":"2025-05-27T12:06:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"62597ea8-60db-4105-929f-37527c6f929c","html_url":"https://github.com/heethesh/SALSA-Semantic-Assisted-SLAM","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/heethesh/SALSA-Semantic-Assisted-SLAM","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heethesh%2FSALSA-Semantic-Assisted-SLAM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heethesh%2FSALSA-Semantic-Assisted-SLAM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heethesh%2FSALSA-Semantic-Assisted-SLAM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heethesh%2FSALSA-Semantic-Assisted-SLAM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/heethesh","download_url":"https://codeload.github.com/heethesh/SALSA-Semantic-Assisted-SLAM/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heethesh%2FSALSA-Semantic-Assisted-SLAM/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259366856,"owners_count":22846799,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["indoor-environments","lifelong-slam","loop-closure","mask-rcnn","openloris-scene","orb-slam2","salsa","semantic-slam","slam","slam-methods","tum-rgbd"],"created_at":"2024-12-25T14:42:51.474Z","updated_at":"2025-06-21T04:38:00.244Z","avatar_url":"https://github.com/heethesh.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![ROS Distro: Melodic](https://img.shields.io/badge/ROS-Melodic-blue.svg)](http://wiki.ros.org/melodic)\n[![CI](https://github.com/heethesh/SLAM-Project/workflows/Lint/badge.svg)](https://github.com/heethesh/SLAM-Project/actions)\n[![License: BSD](https://img.shields.io/badge/License-BSD-yellow.svg)](./LICENSE)\n\n# SALSA: Semantic Assisted Life-Long SLAM for Indoor Environments\n\nWe propose a learning augmented lifelong SLAM method for indoor environments. Most of the existing SLAM methods assume a static environment and disregard dynamic objects. Another problem is that most feature and semantic based SLAM methods fail in repetitive environments. The unexpected changes of surroundings corrupts the quality of the tracking and leads to system failure. This project aims to use learning methods to classify landmarks and objects as dynamic and/or repeatable in nature to better handle optimization, achieve robust performance in a changing environment, and to re-localize in a lifelong-setting. We propose using semantic information and assigning scores to object feature points based on their probability to be dynamic and/or repeatable. We update the front-end, optimization cost functions, and BOW feature generation for loop closures from the original [ORB-SLAM2 pipeline](https://github.com/appliedAI-Initiative/orb_slam_2_ros). Please see [our paper](docs/report.pdf) for more details.\n\n\u003cimg src=\"docs/scores-table.jpg?raw=true\" width=\"450\"\u003e\nAll 80 classes from the COCO dataset are assigned scores. This list is only a subset of some interesting objects. Some objects are exclusively labelled as dynamic objects.\n\n\u0026nbsp;  \n\u0026nbsp;\n\u003cimg src=\"docs/scoremaps.jpg?raw=true\" width=\"500\"\u003e\n\nMask-RCNN was used to segment object instances and the scores were mapped according to the scale shown on the right. The left column shows score maps for the OpenLORIS-Scene [3] cafe1-2 sequence and the right column shows the score maps for TUM-RGBD  walking_static sequence.\n\n## Setup and Usage\nThe current version does not perform segmentation online, the score maps are pre-computed using Detectron 2's Mask RCNN network and appended into the ROS bag file (using `scripts/update_rosbag.py`). See [USAGE.md](docs/USAGE.md) for more details on setting up and running this pipeline.\n\n```\ncatkin build\nroslaunch orb_slam_2_ros salsa.launch\n```\n\nPlease see our report for the changes that we have made in the pipeline. We have tested on monocular camera data, RGB-D camera is still experimental and might work. Stereo mode is supported in the back-end optimization but the front-end (feature culling and ROS node integration) is not fully supported yet.\n\n## Results\n\n#### RMSE Absolute Trajectory Error (ATE) on TUM-RGBD Dataset\n| Sequences      | ORB-SLAM2 [1] | DS-SLAM [2] | SALSA (Ours) |\n|----------------|---------------|-------------|--------------|\n| walking_static | 0.4030m       | 0.0081m     | **0.0059m**      |\n| walking_xyz    | 0.1780m       | **0.0247m**     | 0.0521m      |\n\n#### RMSE Relative Pose Error (RPE) on TUM-RGBD Dataset\n| Sequences      | ORB-SLAM2 [1] | DS-SLAM [2] | SALSA (Ours) |\n|----------------|---------------|-------------|--------------|\n| walking_static | 0.2162m       | 0.0102      | **0.00829m**     |\n| walking_xyz    | 0.4124m       | 0.0333      | **0.02951m**     |\n\n#### RMSE Absolute Trajectory Error (ATE) on OpenLORIS-Scene [3] Dataset\n| Sequences | ORB-SLAM2 [1] | SALSA (Ours) |\n|-----------|-----------|-------------|\n| cafe1_1   | 0.0777m   | **0.0464m**     |\n| cafe1_2   | 0.0813m   | **0.0588m**     |\n\n\u003cimg src=\"docs/results.png?raw=true\"\u003e\nResults of tracking on OpenLORIS-Scene [3] cafe1-1 sequence, TUM-RGBD walking_static and walking_xyz sequences. The columns depict the ground truth trajectory, estimated trajectory results, and tracking and mapping results visualized in Rviz.\n\n## Videos\nTUM-RGBD Sequences: [YouTube Link](https://youtu.be/0TvNcxreGtI)  \nOpenLORIS-Scene Sequences: [YouTube Link](https://youtu.be/9ku1nIotTUw).  \n\u003cbr\u003e\n\u003cimg src=\"docs/tracking.png?raw=true\"\u003e\nDynamic feature points in the red mask region are removed. Possibly dynamic and repeatable feature points have a shade a green/blue and rest of the static object features points are labelled in yellow.\n\n## Citation\n\nPlease cite [our work](docs/report.pdf) if you use SALSA.\n\n```\n@unpublished{SALSA2020,\n    author    = {Ayush Jhalani and Heethesh Vhavle and Sachit Mahajan},\n    title     = {{SALSA}: Semantic Assisted Life-Long {SLAM} for Indoor Environments},\n    note      = {16-833 Robot Localization and Mapping (Spring 2020) Final Project at Carnegie Mellon University},\n    month     = \"May\",\n    year      = \"2020\"\n}\n```\n\n## References\n[1] R. Mur-Artal and J. D. Tardos, \"Orb-slam2: An open-source slam\" system for monocular, stereo, and rgb-d cameras,” IEEE Transactions on Robotics, vol. 33, no. 5, pp. 1255–1262, 2017.  \n\n[2] C. Yu, Z. Liu, X. Liu, F. Xie, Y. Yang, Q. Wei, and Q. Fei, \"Dsslam: A semantic visual slam towards dynamic environments,\" in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, pp. 1168–1174.  \n\n3] X. Shi, D. Li, P. Zhao, Q. Tian, Y. Tian, Q. Long, C. Zhu, J. Song, F. Qiao, L. Song, Y. Guo, Z. Wang, Y. Zhang, B. Qin, W. Yang, F. Wang, R. H. M. Chan, and Q. She, \"Are we ready for service robots? the openloris-scene datasets for lifelong slam,\" 2019. [Dataset Link](https://lifelong-robotic-vision.github.io/dataset/scene)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fheethesh%2Fsalsa-semantic-assisted-slam","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fheethesh%2Fsalsa-semantic-assisted-slam","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fheethesh%2Fsalsa-semantic-assisted-slam/lists"}