Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/nvidia-isaac-ros/isaac_ros_object_detection

NVIDIA-accelerated, deep learned model support for image space object detection
https://github.com/nvidia-isaac-ros/isaac_ros_object_detection

deep-learning gpu inference jetson machine-learning nvidia object-detection ros ros2 ros2-humble tensorrt triton

Last synced: 4 months ago
JSON representation

NVIDIA-accelerated, deep learned model support for image space object detection

Awesome Lists containing this project

README 

          
# Isaac ROS Object Detection



NVIDIA-accelerated, deep learned model support for object detection including DetectNet.





original image

bounding box predictions using DetectNet




## Overview



Isaac ROS Object Detection contains ROS 2 packages to perform object

detection.

`isaac_ros_rtdetr`, `isaac_ros_detectnet`, `isaac_ros_yolov8`, and `isaac_ros_grounding_dino` each provide a method for spatial

classification using bounding boxes with an input image. Classification

is performed by a GPU-accelerated model of the appropriate architecture:



- `isaac_ros_rtdetr`: [RT-DETR models](https://nvidia-isaac-ros.github.io/concepts/object_detection/rtdetr/index.html)

- `isaac_ros_detectnet`: [DetectNet models](https://nvidia-isaac-ros.github.io/concepts/object_detection/detectnet/index.html)

- `isaac_ros_yolov8`: [YOLOv8 models](https://nvidia-isaac-ros.github.io/concepts/object_detection/yolov8/index.html)

- `isaac_ros_grounding_dino`: [Grounding DINO models](https://nvidia-isaac-ros.github.io/concepts/object_detection/grounding_dino/index.html)



The output prediction can be used by perception functions to

understand the presence and spatial location of an object in an image.



image



Each Isaac ROS Object Detection package is used in a graph of nodes to provide a

bounding box detection array with object classes from an input image. A

trained model of the appropriate architecture is required to produce the detection array.



Input images may need to be cropped and resized to maintain the aspect ratio and match the

input resolution of the specific object detection model; image resolution may be reduced to

improve DNN inference performance, which typically scales directly with

the number of pixels in the image. `isaac_ros_dnn_image_encoder`

provides DNN encoder utilities to process the input image into Tensors for the

object detection models.

Prediction results are decoded in model-specific ways,

often involving clustering and thresholding to group multiple detections

on the same object and reduce spurious detections.

Output is provided as a detection array with object classes.



DNNs have a minimum number of pixels that need to be visible on the

object to provide a classification prediction. If a person cannot see

the object in the image, it’s unlikely the DNN will. Reducing input

resolution to reduce compute may reduce what is detected in the image.

For example, a 1920x1080 image containing a distant person occupying 1k

pixels (64x16) would have 0.25K pixels (32x8) when downscaled by 1/2 in

both X and Y. The DNN may detect the person with the original input

image, which provides 1K pixels for the person, and fail to detect the

same person in the downscaled resolution, which only provides 0.25K

pixels for the person.



image



Object detection classifies a rectangle of pixels as containing an

object, whereas image segmentation provides more information and uses

more compute to produce a classification per pixel. Object detection is

used to know if, and where in a 2D image, the object exists. If a 3D

spacial understanding or size of an object in pixels is required, use

image segmentation.



## Isaac ROS NITROS Acceleration



This package is powered by [NVIDIA Isaac Transport for ROS (NITROS)](https://developer.nvidia.com/blog/improve-perception-performance-for-ros-2-applications-with-nvidia-isaac-transport-for-ros/), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.



## Performance



| Sample Graph

                                                                                                                                                                                                    | Input Size

   | AGX Thor T5000

                                                                                                                                                       | AGX Thor T4000

                                                                                                                                                         | DGX Spark

                                                                                                                                                             | x86_64 w/ RTX 5090

                                                                                                                                                  |

|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

| [DetectNet Object Detection Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/benchmarks/isaac_ros_detectnet_benchmark/scripts/isaac_ros_detectnet_graph.py)

                      | 544p

         | [143 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_detectnet_graph-agx_thor.json)


18 ms @ 30Hz

       | [111 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_detectnet_graph-thor-t4000.json)


43 ms @ 30Hz

       | [88.0 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_detectnet_graph-dgx_spark.json)


19 ms @ 30Hz

      | [227 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_detectnet_graph-x86-5090.json)


18 ms @ 30Hz

      |

| [Grounding DINO Object Detection Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/benchmarks/isaac_ros_grounding_dino_benchmark/scripts/isaac_ros_grounding_dino_graph.py)

       | 544p

         | [23.4 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_grounding_dino_graph-agx_thor.json)


50 ms @ 30Hz

 | [22.3 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_grounding_dino_graph-thor-t4000.json)


90 ms @ 30Hz

 | [17.5 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_grounding_dino_graph-dgx_spark.json)


63 ms @ 30Hz

 | [130 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_grounding_dino_graph-x86-5090.json)


15 ms @ 30Hz

 |

| [RT-DETR Object Detection Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/benchmarks/isaac_ros_rtdetr_benchmark/scripts/isaac_ros_rtdetr_graph.py)


SyntheticaDETR

 | 720p

         | [188 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_rtdetr_graph-agx_thor.json)


12 ms @ 30Hz

          | [156 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_rtdetr_graph-thor-t4000.json)


35 ms @ 30Hz

          | [165 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_rtdetr_graph-dgx_spark.json)


12 ms @ 30Hz

          | [444 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.3/results/isaac_ros_rtdetr_graph-x86-5090.json)


11 ms @ 30Hz

         |



---



## Documentation



Please visit the [Isaac ROS Documentation](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/index.html) to learn how to use this repository.



---



## Packages



* [`isaac_ros_detectnet`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html)

  * [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html#quickstart)

  * [Try More Examples](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html#try-more-examples)

  * [ROS 2 Graph Configuration](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html#ros-2-graph-configuration)

  * [Troubleshooting](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html#troubleshooting)

  * [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html#api)

* [`isaac_ros_grounding_dino`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_grounding_dino/index.html)

  * [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_grounding_dino/index.html#quickstart)

  * [Troubleshooting](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_grounding_dino/index.html#troubleshooting)

  * [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_grounding_dino/index.html#api)

* [`isaac_ros_rtdetr`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_rtdetr/index.html)

  * [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_rtdetr/index.html#quickstart)

  * [Try More Examples](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_rtdetr/index.html#try-more-examples)

  * [Troubleshooting](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_rtdetr/index.html#troubleshooting)

  * [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_rtdetr/index.html#api)

* [`isaac_ros_yolov8`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_yolov8/index.html)

  * [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_yolov8/index.html#quickstart)

  * [Troubleshooting](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_yolov8/index.html#troubleshooting)

  * [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_yolov8/index.html#api)

  * [Usage](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_yolov8/index.html#usage)



## Latest



Update 2026-03-23: Introduced early-stage support for SIPL camera framework and LI Eagle stereo CoE/HSB camera with ROS