Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/hridaybavle/semantic_slam
This package is for running the semantic SLAM algorithm using extracted planar surfaces from the received detection
https://github.com/hridaybavle/semantic_slam
Last synced: 6 days ago
JSON representation
This package is for running the semantic SLAM algorithm using extracted planar surfaces from the received detection
- Host: GitHub
- URL: https://github.com/hridaybavle/semantic_slam
- Owner: hridaybavle
- Created: 2020-05-21T07:14:08.000Z (over 4 years ago)
- Default Branch: melodic-devel
- Last Pushed: 2021-07-02T07:46:04.000Z (over 3 years ago)
- Last Synced: 2024-08-02T07:07:38.870Z (3 months ago)
- Language: C++
- Homepage:
- Size: 2.6 MB
- Stars: 143
- Watchers: 5
- Forks: 33
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- Awesome-SLAM - Semantic SLAM
README
# Semantic SLAM #
This package can perform optimization of pose estimated from VO/VIO methods which tend to drift over time. It uses planar surfaces extracted from object detections in order to create a sparse semantic map of the environment, thus optimizing the drift of the VO/VIO algorithms.
### In order to run this package you will need two additional modules ###
- A VO/VIO algorithm: [ROVIO](https://github.com/ethz-asl/rovio), [VINS_MONO](https://github.com/HKUST-Aerial-Robotics/VINS-Mono), [OKVIS](https://github.com/ethz-asl/okvis) etc.
- An Object Detector: [Yolo](https://github.com/leggedrobotics/darknet_ros), [Shape Color Detector](https://bitbucket.org/hridaybavle/bucket_detector)
Currently it can extract planar surfaces and create a semantic map from from the following objects:
- chair
- tvmonitor
- book
- keyboard
- laptop
- bucket
- car
### Related Paper: ###
```latex
@ARTICLE{9045978,
author={Bavle, Hriday and De La Puente, Paloma and How, Jonathan P. and Campoy, Pascual},
journal={IEEE Access},
title={VPS-SLAM: Visual Planar Semantic SLAM for Aerial Robotic Systems},
year={2020},
volume={8},
number={},
pages={60704-60718},
doi={10.1109/ACCESS.2020.2983121}}
```
### Video ###
### How do I set it up?
**First install g2o following these instructions (Tested on Kinetic and Melodic Distributions):**
```
- sudo apt-get install ros-$ROS_DISTRO-libg2o
- sudo cp -r /opt/ros/$ROS_DISTRO/lib/libg2o_* /usr/local/lib
- sudo cp -r /opt/ros/$ROS_DISTRO/include/g2o /usr/local/include
```
**Install OctopMap server for map generation capabilities:**
```
- sudo apt install ros-$ROS_DISTRO-octomap*
```
### Try a simple example with pre-recorded VIO pose and a blue bucket detector:
**Create a ros workspace and clone the following packages:**
- Download the rosbag:
```
wget -P ~/Downloads/ https://www.dropbox.com/s/jnywuvcn2m9ubu2/entire_lab_3_rounds.bag
```
- Create a workspace, clone the repo and compile:
```
mkdir -p workspace/ros/semantic_slam_ws/src/ && cd workspace/ros/semantic_slam_ws/src/
git clone https://github.com/hridaybavle/semantic_slam && git clone https://bitbucket.org/hridaybavle/bucket_detector.git
cd .. && catkin build --cmake-args -DCMAKE_BUILD_TYPE=Release
```
- Launch and visualize
```
source devel/setup.bash
roslaunch semantic_SLAM ps_slam_with_snap_pose_bucket_det_lab_data_with_octomap.launch bagfile:=${HOME}/Downloads/entire_lab_3_rounds.bag show_rviz:=true
```
### Using Docker Image
If the code is giving problems with you local machine, you can try the docker image created with the repo and the required settings.
**Download Docker from: [Docker](https://docs.docker.com/engine/install/ubuntu/)**
**Follow the commands to run the algorithm with the docker**
```
docker pull hridaybavle/semantic_slam:v1
docker run --rm -it --net="host" -p 11311:11311 hridaybavle/semantic_slam:v1 bash
cd ~/workspace/ros/semantic_slam_ws/
source devel/setup.bash
roslaunch semantic_SLAM ps_slam_with_snap_pose_bucket_det_lab_data_with_octomap.launch bagfile:=${HOME}/Downloads/entire_lab_3_rounds.bag show_rviz:=false
```
**Open a new terminal and rviz in local machine**
```
cd ~/Downloads/ && wget https://raw.githubusercontent.com/hridaybavle/semantic_slam/master/rviz/graph_semantic_slam.rviz
rviz -d graph_semantic_slam.rviz
```
### Subsribed Topics
- **/SQ04/snap_vislam/vislam/pose** ([geometry_msgs/PoseStamped](http://docs.ros.org/api/geometry_msgs/html/msg/PoseStamped.html))
The default snapdragon VIO pose published in NED in frame. This message can be remapped remapped to any other VO pose message publishing in NED frame. ([See frame conventions](https://en.wikipedia.org/wiki/Axes_conventions))
- **/rovio/odometry** ([geometry_msgs/PoseStamped](http://docs.ros.org/melodic/api/nav_msgs/html/msg/Odometry.html))
The VIO odometry published in ENU frame. Can be remapped to the desired topic name in the launcher.
- **/depth_registered/points** ([sensor_msgs/PointCloud2](http://docs.ros.org/melodic/api/sensor_msgs/html/msg/PointCloud2.html))
The point cloud required for planar surface extraction.
- **/darknet_ros/bounding_boxes**([darknet_msgs_ros/BoundingBoxes](https://github.com/leggedrobotics/darknet_ros))
The detection bounding boxes published by yolo if using the yolo detector ros package.
- **/image_processed/bounding_boxes**([ShapeColor_ObjectDetection/DetectedObjects](https://[email protected]/hridaybavle/bucket_detector.git))
The detection bounding boxes if using the bucket detector. It can be downloaded from the link above.
### Published Topics
- **robot_pose**([geometry_msgs/PoseStamped](http://docs.ros.org/melodic/api/nav_msgs/html/msg/Odometry.html))
The pose of the robot estimated by the algo.
- **robot_path**([nav_msgs/Path](http://docs.ros.org/melodic/api/nav_msgs/html/msg/Path.html))
The path of the robot psoe for visualization.
- **keyframe_poses**([geometry_msgs/PoseArray](http://docs.ros.org/melodic/api/geometry_msgs/html/msg/PoseArray.html))
The poses of the keyframe being added to the g2o graph.
- **mapped_landmarks**([visualization_msgs/MarkerArray](http://docs.ros.org/melodic/api/visualization_msgs/html/msg/MarkerArray.html))
The mapped semantic planar surfaces.
- **detected_landmarks**([visualization_msgs/MarkerArray](http://docs.ros.org/melodic/api/visualization_msgs/html/msg/MarkerArray.html))
The detected landmarks in the current frame.
The configurations of the algorithms can be found inside the cfg folder in order to be changed accordingly.
### Published TFs
- **map to odom transform:** The transform published between the map frame and the odom frame after the corrections from the semantic SLAM.
- **base_link to odom transform:** The transform published between the base_link (on the robot) frame and the odom frame as estimated by the VO/VIO algorithm.