Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/IntelligentRoboticsLabs/gb_visual_detection_3d


https://github.com/IntelligentRoboticsLabs/gb_visual_detection_3d

Last synced: 2 months ago
JSON representation

Lists

README 

        
# Package Summary



[![Build Status](https://travis-ci.com/IntelligentRoboticsLabs/gb_visual_detection_3d.svg?branch=master)](https://travis-ci.com/IntelligentRoboticsLabs/gb_visual_detection_3d)



## darknet_ros_3d



* **Maintainer status:** maintained

* **Maintainer:** Fernando González Ramos 

* **Author:** Francisco Martín Rico ; Fernando González Ramos 

* **License:** BSD

* **Source:** [Github](https://github.com/IntelligentRoboticsLabs/gb_visual_detection_3d)



### Contents

* Overview

  * Previous steps

* Quick start

  * Instalation

  * How it works

* Nodes

  * darknet3d_node



## Overview

darknet_ros_3d provides you 3d bounding boxes of the objects contained in an objects list, where is specificated the 3d position of each object.

Using a RGBD camera like *Asus Xtion*, *Orbbec Astra* or *Intel Realsense* and neuronal network *darknet ros*, objects can be detected and his position can be calculated.



In addition, there is a visual debugger tool based on visual markers that you can see with rviz



![Image](https://github.com/IntelligentRoboticsLabs/gb_visual_detection_3d/raw/master/docs/visual_markers.png)



### Previous Steps



The first we have to know is that *darknet_ros_3d* package have dependencies as follow:



* rclcpp

* rclcpp_lifecycle

* darknet_ros_msgs

* [gb_visual_detection_3d_msgs](https://github.com/IntelligentRoboticsLabs/gb_visual_detection_3d_msgs)

* sensor_msgs

* tf2

* tf2_ros

* tf2_sensor_msgs

* visualization_msgs

* eigen3_cmake_module

* Eigen3



**You can install Darknet Ros following** [this steps](https://github.com/leggedrobotics/darknet_ros). **NOTE:** ros2 branch



## Quick Start



### Instalation



You must to clone *gb_visual_detection_3d* into *src* folder located in your workspace. Later, you have to compile it typing ``colcon build``.



### How It Works



First of all, is necessary to run camera driver. **Make sure that camera driver is publishing point cloud information**.



Later, you must run *darknet_ros* and, if everything worked properly, you should see 2d bounding boxes in your screen. If not, you have a problem with *darknet_ros* package.



Now, you can run *darknet_ros_3d* typing ``ros2 launch darknet_ros_3d darknet_ros_3d.launch.py``. If you want to change default parameters like topics it subscribe, you can change it in the configuration file located at ``~/catkin_ws/src/gb_visual_detection_3d/darknet_ros_3d/darknet_ros_3d/config/``. Default parameters are the following:



* **interested_classes:** Classes you want to detect. It must be classes names than exists previously in darknet ros.



* **mininum_detection_threshold:** Maximum distance range between any pixel of image and the center pixel of the image to be considered.



* **minimum_probability:** Minimum object probability (provided by *darknet_ros*) to be considered.



* **darknet_ros_topic:** topic where darknet_ros publicates it's bounding boxes. ``/darknet_ros/bounding_boxes``.



* **point_cloud_topic:** topic where point cloud is published from camera. By default: ``/camera/pointcloud``. **It is important that point cloud topic be of PointCloud2 type and it be depth_registered**.



* **working_frame:** frame that all measurements are based on. By default, *camera_link*. **It is very important that if you want to change this frame, it has the same axes than camera_link**, if you would want 3d coordinates in another axis, you must change it later (once 3d bounding box has been calculated).



## Nodes



### darknet3d_node



*darknet3d_node* provide bounding boxes. This bounding boxes are combinated with point cloud information to calculate (xmin, ymin, zmin) and (xmax, ymax, zmax) 3D coordinates.



Then, *darknet_ros_3d* publicates it's own bounding boxes array of *BoundingBoxes3d* type in ``/darknet_ros_3d/bounding_boxes_3d`` by default, which is an array of *BoundingBox3d* that contains the following information:

```

std_msgs/Header header

BoundingBox3d[] bounding_boxes

```



**BoundingBox3d:**



```

string object_name

float64 probability

float64 xmin

float64 ymin

float64 xmax

float64 ymax

float64 zmin

float64 zmax

```

* **object_name:** Object name.

* **probability:** Probability of certainty.

* **xmin:** X coordinate in meters of left upper corner of bounding box.

* **xmax:** X coordinate in meters of right lower corner of bounding box.

* **ymin:** Y coordinate in meters of left upper corner of bounding box.

* **ymax:** Y coordinate in meters of right lower corner of bounding box.

* **zmin:** Z coordinate in meters of nearest pixel of bounding box.

* **zmax:** Z coordinate in meters of the furthest pixel of bounding box.



**An example of output is as follow:**



```

header:

  stamp:

    sec: 1593723845

    nanosec: 430724839

  frame_id: camera_link

bounding_boxes:

- object_name: person

  probability: 0.7609682679176331

  xmin: 0.4506256580352783

  ymin: -0.3164764642715454

  xmax: 0.7936256527900696

  ymax: 0.11368180811405182

  zmin: -0.25958430767059326

  zmax: 0.10506562888622284



```



You can visualize the markers on *rviz* adding **MarkerArray** and subscribing to topic ``/darknet_ros_3d/markers``.