https://github.com/flexrobotics/roboflex_realsense
Roboflex support for broadcasting rgb and depth messages from the realsense camera.
https://github.com/flexrobotics/roboflex_realsense
realsense2 roboflex
Last synced: 6 months ago
JSON representation
Roboflex support for broadcasting rgb and depth messages from the realsense camera.
- Host: GitHub
- URL: https://github.com/flexrobotics/roboflex_realsense
- Owner: flexrobotics
- License: mit
- Created: 2023-10-19T21:22:34.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-03-06T21:59:46.000Z (about 1 year ago)
- Last Synced: 2024-11-06T00:28:15.707Z (7 months ago)
- Topics: realsense2, roboflex
- Language: C++
- Homepage: https://github.com/flexrobotics/roboflex_realsense
- Size: 34.2 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# roboflex.realsense
Support for the Realsense D435 camera. We really just wrap the sdk and encode data into messages. It might work with other D4** cameras - YMMV.
## System Dependencies
Sort of: none. We use cmake to build. If find_package(realsense2) fails (meaning cmake can't find some system-installed version of realsense2), we fetch and build it.
## Install
pip install roboflex.realsense
## Import
import roboflex.realsense as rr
## Nodes
There is only one: **RealsenseSensor**
You can use a class method to discover the serial numbers of all the cameras currently connected to your computer:
rr.RealsenseSensor.get_connected_device_serial_numbers()
There are two ways to instantiate realsense sensors:
1. Via the constructor - you must know the serial number. See below for documentation of the Config type.
sensor = rr.RealsenseSensor(
serial_number: str,
config: rr.Config,
name: str = "RealsenseSensor",
)
2. Via a class method - if you have a single realsense attached,
this is the easiest way - you don't even have to know the serial number.
sensor = rr.RealsenseSensor.get_one_sensor(
config: rr.Config,
name: str = "RealsenseSensor",
)
Use it like so:
# must be started!
sensor.start()
# You can get the serial number:
sensor.serial_number -> str
# You can get the configuration back:
sensor.config -> rr.Config
# You can get the two camera-k matrices:
sensor.depth_camera_k -> np.array
sensor.color_camera_k -> np.array
# You can get various device values:
sensor.width_pixels_color -> int
sensor.height_pixels_color -> int
sensor.width_pixels_depth -> int
sensor.height_pixels_depth -> int
sensor.fps_color -> int
sensor.fps_depth -> int
# You can manually trigger a message event:
# (for the most part you won't need this - only
# if you have special need to control the message
# production), and don't want to use "start".
sensor.produce()
# You can turn the IR laser off and on
sensor.set_laser_on_off(False)
sensor.set_laser_on_off(True)
## Messages
RealsenseSensor produces a single message type: **RealsenseFrameset**.
from roboflex.realsense import RealsenseFrameset
API:
# the timestamp just before reading from device
message.t0 -> Float
# the timestamp just after reading from device
message.t1 -> Float
# numpy array of shape=(height, width, 3), dtype=uint8
message.rgb -> np.ndarray
# numpy array of shape=(height, width), dtype=uint16
message.depth -> np.ndarray
# numpy array of shape=(height, width), dtype=uint8
message.ir1 -> np.ndarray
# numpy array of shape=(height, width), dtype=uint8
message.ir2 -> np.ndarray
# which camera (if any) the frame is aligned to
message.aligned_to -> rr.CameraType
# the serial number of the camera that produced this frameset
message.serial_number -> str
# the color camera k of the camera that produced this frameset
message.camera_k_rgb -> numpy array of (3, 3)
# the depth camera k of the camera that produced this frameset
message.camera_k_depth -> numpy array of (3, 3)
# the ir1 camera k of the camera that produced this frameset
message.camera_k_ir1 -> numpy array of (3, 3)
# the ir2 camera k of the camera that produced this frameset
message.camera_k_ir2 -> numpy array of (3, 3)
# the index, from the device, of this frameset
message.frame_number -> int
# the timestamp, from the device, of this frameset, in epoch seconds
message.timestamp -> float
DYNOFLEX:
d = DynoFlex.from_msg(message)
# the timestamp just before reading from device
d["t0"] -> Double
# the timestamp just after reading from device
d["t1"] -> Double
# numpy array of shape=(height, width, 3), dtype=uint8
d["rgb"] -> np.ndarray
# numpy array of shape=(height, width), dtype=uint16
d["depth"] -> np.ndarray
# numpy array of shape=(height, width), dtype=uint8
d["ir1"] -> np.ndarray
# numpy array of shape=(height, width), dtype=uint8
d["ir2"] -> np.ndarray
# which camera (if any) the frame is aligned to
d["aligned_to"] -> rr.CameraType
# the serial number of the camera that produced this frameset
d["serial_number"] -> str
# the color camera k of the camera that produced this frameset
d["camera_k_rgb"] -> numpy array of (3, 3)
# the depth camera k of the camera that produced this frameset
d["camera_k_depth"] -> numpy array of (3, 3)
# the camera k of the infrared camera 1 that produced this frameset
d["camera_k_ir1"] -> numpy array of (3, 3)
# the camera k of the infrared camera 2 that produced this frameset
d["camera_k_ir2"] -> numpy array of (3, 3)
# the index, from the device, of this frameset
d["n"] -> int
# the timestamp, from the device, of this frameset, in epoch seconds
d["t"] -> float
## Other
Some types used for configuration
Some enums:
CameraType:
RGB,
DEPTH,
IR1, # raw infrared camera1
IR2 # raw infrared camera2
# You can OR these together like so:
my_cameras = camera_type_or([CameraType.RGB, CameraType.DEPTH, etc])
# You can test for camera types:
has_rgb = camera_type_contains(my_camera_type, CameraType.RGB)
CameraAlignment:
NONE,
RGB,
DEPTH
D400VisualPreset:
CUSTOM,
DEFAULT,
HAND,
HIGH_ACCURACY,
HIGH_DENSITY,
MEDIUM_DENSITY
TemporalFilterParameters:
f = rr.TemporalFilterParameters(
alpha: float = 0.4,
delta: float = 20.0,
persistance_control: int = 7,
)
f.alpha -> float
f.delta -> float
f.persistance_control -> int
Config:
# all paramters are optional - defaults shown below
c = rr.Config(
camera_type: rr.CameraType = CameraType.RGB | CameraType.DEPTH,
align_to: rr.CameraAlignment = rr.CameraAlignment.NONE,
prioritize_ae: Bool = False,
rgb_settings: Dict[str, int] = {
"fps": 0,
"width": 0,
"height": 0,
},
depth_settings: Dict[str, int] = {
"fps": 0,
"width": 0,
"height": 0,
},
depth_visual_preset: rr.D400VisualPreset = rr.D400VisualPreset.DEFAULT,
temporal_filter_parameters: rr.TemporalFilterParameters = None,
hole_filling_mode: Optional[int] = None,
decimation_filter: Optional[int] = None,
)
# which actual cameras are in use - this a bitmask, OR-ed together
c.camera_type -> rr.CameraType
# which camera the frames will be aligned to (if any)
c.align_to -> rr.CameraAligment
# When `true`, allows fps to drop in order to better expose
# frames, such as in dimly lit environments
c.prioritize_ae -> Bool
# fps, height, and width of the color camera
c.rgb_settings -> Dict[str, int]
# fps, height, and width of the depth camera
c.depth_settings -> Dict[str, int]
# the depth camera visual preset
c.depth_visual_preset -> rr.D400VisualPreset
# the temporal filter, if any - can be None
c.temporal_filter_parameters -> rr.TemporalFilterParameters
# the 'hole filling mode', if any - can be None
# 0: fill_from_left
# 1: farthest_from_around
# 2: nearest_from_around
c.hole_filling_mode -> int
# NOTE!
The realsense camera supports many more settings, such as laser power, etc. If there's something you want supported that's not
here, just let me know.