https://github.com/nvidia-isaac-ros/isaac_ros_freespace_segmentation
NVIDIA-accelerated, deep-learned freespace segmentation
https://github.com/nvidia-isaac-ros/isaac_ros_freespace_segmentation
deep-learning deep-neural-networks freespace gpu jetson nvidia ros2 ros2-humble
Last synced: 9 months ago
JSON representation
NVIDIA-accelerated, deep-learned freespace segmentation
- Host: GitHub
- URL: https://github.com/nvidia-isaac-ros/isaac_ros_freespace_segmentation
- Owner: NVIDIA-ISAAC-ROS
- License: apache-2.0
- Created: 2023-03-28T22:12:25.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2025-02-28T01:47:14.000Z (over 1 year ago)
- Last Synced: 2025-06-29T19:39:47.074Z (about 1 year ago)
- Topics: deep-learning, deep-neural-networks, freespace, gpu, jetson, nvidia, ros2, ros2-humble
- Language: C++
- Homepage: https://developer.nvidia.com/isaac-ros-gems
- Size: 90.8 KB
- Stars: 40
- Watchers: 2
- Forks: 4
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# Isaac ROS Freespace Segmentation
NVIDIA-accelerated, deep-learned freespace segmentation
## Overview
Isaac ROS Freespace Segmentation contains an ROS 2 package to produce
occupancy grids for navigation. By processing a freespace segmentation
mask with the pose of the robot relative to the ground, Bi3D Freespace
produces an occupancy grid for
[Nav2](https://github.com/ros-planning/navigation2), which is used to
avoid obstacles during navigation. This package is GPU accelerated to
provide real-time, low latency results in a robotics application. Bi3D
Freespace provides an additional occupancy grid source for mobile robots
(ground based).
`isaac_ros_bi3d` is used in a graph of nodes to provide a freespace
segmentation mask as one output from a time-synchronized input left and
right stereo image pair. The freespace mask is used by
`isaac_ros_bi3d_freespace` with TF pose of the camera relative to the
ground to compute planar freespace into an occupancy grid as input to
[Nav2](https://github.com/ros-planning/navigation2).
There are multiple methods to predict the occupancy grid as an input to
navigation. None of these methods are perfect; each has limitations on
the accuracy of its estimate from the sensor providing measured
observations. Each sensor has a unique field of view, range to provide
its measured view of the world, and corresponding areas it does not
measure. Bi3D Freespace provides a diverse approach to
identifying obstacles from freespace. Stereo camera input used for this
function is diverse relative to lidar, and has a better vertical field
of view than most lidar units, allowing for perception of low lying
obstacles that lidar can miss. Bi3D Freespace provides a
robust, vision-based complement to lidar occupancy scanning.
## Isaac ROS NITROS Acceleration
This package is powered by [NVIDIA Isaac Transport for ROS (NITROS)](https://developer.nvidia.com/blog/improve-perception-performance-for-ros-2-applications-with-nvidia-isaac-transport-for-ros/), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.
## Performance
| Sample Graph
| Input Size
| AGX Orin
| Orin NX
| Orin Nano Super 8GB
| x86_64 w/ RTX 4090
|
|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Freespace Segmentation Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/benchmarks/isaac_ros_bi3d_freespace_benchmark/scripts/isaac_ros_bi3d_fs_node.py)
| 576p
| [3340 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_fs_node-agx_orin.json)
1.7 ms @ 30Hz
| [2530 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_fs_node-orin_nx.json)
1.5 ms @ 30Hz
| [2140 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_fs_node-orin_nano.json)
1.9 ms @ 30Hz
| [3500 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_fs_node-x86-4090.json)
0.44 ms @ 30Hz
|
| [Freespace Segmentation Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/benchmarks/isaac_ros_bi3d_freespace_benchmark/scripts/isaac_ros_bi3d_fs_graph.py)
| 576p
| [40.3 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_fs_graph-agx_orin.json)
79 ms @ 30Hz
| [27.6 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_fs_graph-orin_nx.json)
98 ms @ 30Hz
| [31.8 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_fs_graph-orin_nano.json)
55 ms @ 30Hz
| [102 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_fs_graph-x86-4090.json)
30 ms @ 30Hz
|
---
## Documentation
Please visit the [Isaac ROS Documentation](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_freespace_segmentation/index.html) to learn how to use this repository.
---
## Packages
* [`isaac_ros_bi3d_freespace`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_freespace_segmentation/isaac_ros_bi3d_freespace/index.html)
* [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_freespace_segmentation/isaac_ros_bi3d_freespace/index.html#quickstart)
* [Try More Examples](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_freespace_segmentation/isaac_ros_bi3d_freespace/index.html#try-more-examples)
* [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_freespace_segmentation/isaac_ros_bi3d_freespace/index.html#api)
## Latest
Update 2024-12-10: Update to be compatible with JetPack 6.1