Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Alpaca-zip/ultralytics_ros
ROS/ROS 2 package for Ultralytics YOLOv8 real-time object detection and segmentation. https://github.com/ultralytics/ultralytics
https://github.com/Alpaca-zip/ultralytics_ros
object-detection object-segmentation ros ros2 ultralytics yolo yolov3 yolov5 yolov8
Last synced: 12 days ago
JSON representation
ROS/ROS 2 package for Ultralytics YOLOv8 real-time object detection and segmentation. https://github.com/ultralytics/ultralytics
- Host: GitHub
- URL: https://github.com/Alpaca-zip/ultralytics_ros
- Owner: Alpaca-zip
- License: agpl-3.0
- Created: 2023-05-27T16:27:22.000Z (about 1 year ago)
- Default Branch: humble-devel
- Last Pushed: 2024-04-12T14:14:22.000Z (3 months ago)
- Last Synced: 2024-05-01T15:44:42.820Z (about 2 months ago)
- Topics: object-detection, object-segmentation, ros, ros2, ultralytics, yolo, yolov3, yolov5, yolov8
- Language: C++
- Homepage:
- Size: 205 KB
- Stars: 142
- Watchers: 2
- Forks: 26
- Open Issues: 13
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Citation: CITATION.cff
Lists
- awesome-yolo-object-detection - Alpaca-zip/ultralytics_ros - zip/ultralytics_ros?style=social"/> : ROS/ROS2 package for Ultralytics YOLOv8 real-time object detection. (Other Versions of YOLO)
- awesome-stars - Alpaca-zip/ultralytics_ros - ROS/ROS 2 package for Ultralytics YOLOv8 real-time object detection and segmentation. https://github.com/ultralytics/ultralytics (C++)
README
# ultralytics_ros
### Introduction
ROS/ROS 2 package for real-time object detection and segmentation using the Ultralytics YOLO, enabling flexible integration with various robotics applications.
| `tracker_node` | `tracker_with_cloud_node` |
| :------------: | :-----------------------: |
| 
| 
|
- The `tracker_node` provides real-time object detection on incoming ROS/ROS 2 image messages using the Ultralytics YOLO model.
- The `tracker_with_cloud_node` provides functionality for 3D object detection by integrating 2D detections, mask image, LiDAR data, and camera information.
### Status
| ROS distro | Industrial CI | Docker |
| :--------: | :-----------: | :----: |
| ROS Melodic | [](https://github.com/Alpaca-zip/ultralytics_ros/actions/workflows/melodic-ci.yml) | [](https://github.com/Alpaca-zip/ultralytics_ros/actions/workflows/melodic-docker-build-check.yml)
| ROS Noetic | [](https://github.com/Alpaca-zip/ultralytics_ros/actions/workflows/noetic-ci.yml) | [](https://github.com/Alpaca-zip/ultralytics_ros/actions/workflows/noetic-docker-build-check.yml)
| ROS 2 Humble | [](https://github.com/Alpaca-zip/ultralytics_ros/actions/workflows/humble-ci.yml) | [](https://github.com/Alpaca-zip/ultralytics_ros/actions/workflows/humble-docker-build-check.yml)
## Setup ⚙
### ROS Melodic
```bash
$ cd ~/{ROS_WORKSPACE}/src
$ GIT_LFS_SKIP_SMUDGE=1 git clone -b melodic-devel https://github.com/Alpaca-zip/ultralytics_ros.git
$ rosdep install -r -y -i --from-paths .
$ pip install pipenv
$ cd ultralytics_ros
$ pipenv install
$ pipenv shell
$ cd ~/{ROS_WORKSPACE} && catkin build
```
### ROS Noetic
```bash
$ cd ~/{ROS_WORKSPACE}/src
$ GIT_LFS_SKIP_SMUDGE=1 git clone -b noetic-devel https://github.com/Alpaca-zip/ultralytics_ros.git
$ rosdep install -r -y -i --from-paths .
$ python3 -m pip install -r ultralytics_ros/requirements.txt
$ cd ~/{ROS_WORKSPACE} && catkin build
```
### ROS 2 Humble
```bash
$ cd ~/{ROS2_WORKSPACE}/src
$ GIT_LFS_SKIP_SMUDGE=1 git clone -b humble-devel https://github.com/Alpaca-zip/ultralytics_ros.git
$ rosdep install -r -y -i --from-paths .
$ python3 -m pip install -r ultralytics_ros/requirements.txt
$ cd ~/{ROS2_WORKSPACE} && $ colcon build
```
**NOTE**: If you want to download KITTI datasets, remove `GIT_LFS_SKIP_SMUDGE=1` from the command line.
## Run 🚀
### ROS Melodic & ROS Noetic
**`tracker_node`**
```bash
$ roslaunch ultralytics_ros tracker.launch debug:=true
```
**`tracker_node` & `tracker_with_cloud_node`**
```bash
$ roslaunch ultralytics_ros tracker_with_cloud.launch debug:=true
```
### ROS 2 Humble
**`tracker_node`**
```bash
$ ros2 launch ultralytics_ros tracker.launch.xml debug:=true
```
**`tracker_node` & `tracker_with_cloud_node`**
```bash
$ ros2 launch ultralytics_ros tracker_with_cloud.launch.xml debug:=true
```
**NOTE**: If the 3D bounding box is not displayed correctly, please consider using a lighter yolo model(`yolov8n.pt`) or increasing the `voxel_leaf_size`.
## `tracker_node`
### Params
- `yolo_model`: Pre-trained Weights.
For yolov8, you can choose `yolov8*.pt`, `yolov8*-seg.pt`.
| YOLOv8 | 
|
| :-------------: | :-------------: |
| **YOLOv8-seg** | 
|
See also: https://docs.ultralytics.com/models/
- `input_topic`: Topic name for input image.
- `result_topic`: Topic name of the custom message containing the 2D bounding box and the mask image.
- `result_image_topic`: Topic name of the image on which the detection and segmentation results are plotted.
- `conf_thres`: Confidence threshold below which boxes will be filtered out.
- `iou_thres`: IoU threshold below which boxes will be filtered out during NMS.
- `max_det`: Maximum number of boxes to keep after NMS.
- `tracker`: Tracking algorithms.
- `device`: Device to run the model on(e.g. cpu or cuda:0).
```xml
```
```xml
```
- `classes`: List of class indices to consider.
```xml
```
See also: https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/coco128.yaml
- `result_conf`: Whether to plot the detection confidence score.
- `result_line_width`: Line width of the bounding boxes.
- `result_font_size`: Font size of the text.
- `result_labels`: Font to use for the text.
- `result_font`: Whether to plot the label of bounding boxes.
- `result_boxes`: Whether to plot the bounding boxes.
### Topics
- Subscribed Topics:
- Image data from `input_topic` parameter. ([sensor_msgs/Image](https://github.com/ros2/common_interfaces/blob/humble/sensor_msgs/msg/Image.msg))
- Published Topics:
- Plotted images to `result_image_topic` parameter. ([sensor_msgs/Image](https://github.com/ros2/common_interfaces/blob/humble/sensor_msgs/msg/Image.msg))
- Detected objects(2D bounding box, mask image) to `result_topic` parameter. (ultralytics_ros/YoloResult)
```
std_msgs/Header header
vision_msgs/Detection2DArray detections
sensor_msgs/Image[] masks
```
## `tracker_with_cloud_node`
### Params
- `camera_info_topic`: Topic name for camera info.
- `lidar_topic`: Topic name for lidar.
- `yolo_result_topic`: Topic name of the custom message containing the 2D bounding box and the mask image.
- `yolo_3d_result_topic`: Topic name for 3D bounding box.
- `cluster_tolerance`: Spatial cluster tolerance as a measure in the L2 Euclidean space.
- `voxel_leaf_size`: Voxel size for pointcloud downsampling.
- `min_cluster_size`: Minimum number of points that a cluster needs to contain.
- `max_cluster_size`: Maximum number of points that a cluster needs to contain.
### Topics
- Subscribed Topics:
- Camera info from `camera_info_topic` parameter. ([sensor_msgs/CameraInfo](https://docs.ros.org/en/api/sensor_msgs/html/msg/CameraInfo.html))
- Lidar data from `lidar_topic` parameter. ([sensor_msgs/PointCloud2](https://docs.ros.org/en/api/sensor_msgs/html/msg/PointCloud2.html))
- Detected objects(2D bounding box, mask image) from `yolo_result_topic` parameter. (ultralytics_ros/YoloResult)
```
std_msgs/Header header
vision_msgs/Detection2DArray detections
sensor_msgs/Image[] masks
```
- Published Topics:
- Detected cloud points to `/detection_cloud` topic. ([sensor_msgs/PointCloud2](https://docs.ros.org/en/api/sensor_msgs/html/msg/PointCloud2.html))
- Detected objects(3D bounding box) to `yolo_3d_result_topic` parameter. ([vision_msgs/Detection3DArray](http://docs.ros.org/en/lunar/api/vision_msgs/html/msg/Detection3DArray.html))
- Visualization markers to `/detection_marker` topic. ([visualization_msgs/MarkerArray](https://docs.ros.org/en/api/visualization_msgs/html/msg/MarkerArray.html))
## Docker with KITTI datasets 🐳
[](https://hub.docker.com/r/alpacazip/ultralytics_ros)
### Docker Pull & Run
**ROS Melodic**
```bash
$ docker pull alpacazip/ultralytics_ros:melodic
$ docker run -p 6080:80 --shm-size=512m alpacazip/ultralytics_ros:melodic
```
**ROS Noetic**
```bash
$ docker pull alpacazip/ultralytics_ros:noetic
$ docker run -p 6080:80 --shm-size=512m alpacazip/ultralytics_ros:noetic
```
**ROS 2 Humble**
```bash
$ docker pull alpacazip/ultralytics_ros:humble
$ docker run -p 6080:80 --shm-size=512m alpacazip/ultralytics_ros:humble
```
### Run tracker_node & tracker_with_cloud_node
**ROS Melodic**
```bash
$ roscd ultralytics_ros && pipenv shell
$ roslaunch ultralytics_ros kitti_predict_with_cloud.launch
$ cd ~/catkin_ws/src/ultralytics_ros/rosbag && rosbag play kitti_2011_09_26_drive_0106_synced.bag --clock --loop
```
**ROS Noetic**
```bash
$ roslaunch ultralytics_ros kitti_tracker_with_cloud.launch
$ cd ~/catkin_ws/src/ultralytics_ros/rosbag && rosbag play kitti_2011_09_26_drive_0106_synced.bag --clock --loop
```
**ROS 2 Humble**
```bash
$ ros2 launch ultralytics_ros kitti_tracker_with_cloud.launch.xml
$ cd ~/colcon_ws/src/ultralytics_ros/ros2bag && ros2 bag play kitti_2011_09_26_drive_0106_synced --clock --loop
```