Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/ros-controls/mujoco_ros2_control

Provides a ROS 2 control hardware interface for the MuJoCo physics simulator, including plugins for various sensors, RGB-D cameras, and lidar.
https://github.com/ros-controls/mujoco_ros2_control

mujoco robotics ros-controls ros2 simulation

Last synced: 28 days ago
JSON representation

Provides a ROS 2 control hardware interface for the MuJoCo physics simulator, including plugins for various sensors, RGB-D cameras, and lidar.

Awesome Lists containing this project

README 

          
# MuJoCo ros2_control Simulation



This package contains a ros2_control system interface for the [MuJoCo Simulator](https://mujoco.readthedocs.io/en/3.4.0/overview.html).

It was originally written for simulating robot hardware in NASA Johnson's [iMETRO facility](https://ntrs.nasa.gov/citations/20230015485).



The system interface wraps MuJoCo's [Simulate App](https://github.com/google-deepmind/mujoco/tree/3.4.0/simulate) to provide included functionality.

Because the app is not bundled as a library, we compile it directly from a local install of MuJoCo.



Parts of this library are also based on the MoveIt [mujoco_ros2_control](https://github.com/moveit/mujoco_ros2_control) package.



## Installation



This interface has only been tested against ROS 2 jazzy and MuJoCo `3.4.0`.

It should also be compatible with kilted and rolling, but we do not actively maintain those.

We assume all required ROS dependencies have been installed either manually or with `rosdep`.



For configuring MuJoCo, the included [CMakeLists.txt](./CMakeLists.txt) will download and install the tarfile automatically.

As long as users have a good network connection there should not be an issue.



However, a local install of MuJoCo can be used to build the application by setting the following environment variables,



```bash

# The tested version

MUJOCO_VERSION=3.4.0



# Wherever it was installed and extracted on your machine

MUJOCO_INSTALL_DIR=/opt/mujoco/mujoco-3.4.0

```



From there the library can be compiled with `colcon build ...`, as normal.



## URDF Model Conversion



Mujoco does not support the full feature set of xacro/URDFs in the ROS 2 ecosystem.

As such, users are required to convert any existing robot description files to an MJCF format.

This includes adding actuators, sensors, and cameras as needed to the MJCF XML.



We have built a *highly experimental tool to automate URDF conversion.

For more information refer to the [documentation](./docs/TOOLS.md).



## Hardware Interface Setup



### Plugin



This application is shipped as a ros2_control hardware interface, and can be configured as such.

Just specify the plugin and point to a valid MJCF on launch:



```xml

  

    

      mujoco_ros2_control/MujocoSystemInterface

      $(find my_description)/description/scene.xml



      

      $(find my_description)/config/pids.yaml



      

      5.0



      

      $(find my_description)/config/start_positions.xml



      

      6.0



      

      10.0



      

      false

    

  ...

```



Due to compatibility issues, we use a [slightly modified ROS 2 control node](./src/mujoco_ros2_control_node.cpp).

It is the same executable and parameters as the upstream, but requires updating the launchfile:



```python

    control_node = Node(

        # Specify the control node from this package!

        package="mujoco_ros2_control",

        executable="ros2_control_node",

        output="both",

        parameters=[

            {"use_sim_time": True},

            controller_parameters,

        ],

    )

```



> [!NOTE]

> We can remove the the ROS 2 control node after the next ros2_control upstream release,

> as the simulation requires [this PR](https://github.com/ros-controls/ros2_control/pull/2654) to run.

> The hardware interface _should_ then be compatible with `humble`, `jazzy`, and `kilted`.



### Joints



Joints in the ros2_control interface are mapped to actuators defined in the MJCF.

The system supports different joint control modes based on the actuator type and available command interfaces.



We rely on MuJoCo's PD-level ctrl input for direct position, velocity, or effort control.

For velocity, motor, or custom actuators, a position or velocity PID is created if specified using ROS parameters to enable accurate control.

Incompatible actuator-interface combinations trigger an error.



Refer to Mujoco's [actuation model](https://mujoco.readthedocs.io/en/stable/computation/index.html#geactuation) for more information.



Of note, only one type of MuJoCo actuator per-joint can be controllable at a time, and the type CANNOT be switched during runtime (i.e., switching from position to motor actuator is not supported).

However, the active command interface can be switched dynamically, allowing control to shift between position, velocity, or effort as supported by the actuator type.

Users are required to manually adjust actuator types and command interfaces to ensure that they are compatible.



For example a position controlled joint on the mujoco



```xml

  

    

  

```



Could map to the following hardware interface:



```xml

  

    

    

    

      0.0

    

    

    

  

```



**Supported modes between MuJoCo actuators and ros2_control command interfaces:**



  

    

      

      

      MuJoCo Actuators

    

    

      

      position

      velocity

      motor, general, etc 

    

  

  

    

      ros2 control
command
interfaces

      position

      Native support

      Supported using PIDs

      Supported using PIDs

    

    

      velocity

      Not supported

      Native support

      Supported using PIDs

    

    

      effort

      Not supported

      Not supported

      Native support

    

  



> [!NOTE]

> The `torque` and `force` command/state interfaces are semantically equivalent to `effort`, and map to the same underlying data in the sim.



Switching actuator/control types on the fly is an [open issue](#13).



### Sensors



The hardware interfaces supports force-torque sensors (FTS) and inertial measurement units (IMUs).

Mujoco does not support modeling complete FTS and IMUs out of the box, so we combine supported MJCF constructs to map to a ros2_control sensor.

The types and other parameters can be specified in the ros2_control xacro, as noted below.



For FTS, we model the `force` and `torque` sensors individually in the MJFC.

For a sensor named `fts_sensor`, we suffix each entry accordingly as `fts_sensor_force` and `fts_sensor_torque`.

For example, in the MJCF,



```xml

  

    

    

  

```



In the corresponding ros2_control xacro, this becomes a single sensor:



```xml

  

    fts

    

    fts_sensor

    

    _force

    

    _torque

    

    

    

    

    

    

  

```



Similarly, for an IMU, we simulate a `framequat`, `gyro`, and `accelerometer` as a single IMU.



```xml

  

      

      

      

  

```



Which then map to the corresponding ros2_control sensor:



```xml

  

    imu

    

    imu_sensor

    

    _quat

    

    _gyro

    

    _accel

    

    

    

    

    

    

    

    

    

    

  

```



These sensor state interfaces can then be used out of the box with the standard broadcasters.



### Cameras



Any `camera` included in the MJCF will automatically have its RGB-D images and info published to ROS topics.

Currently all images are published at a fixed 5hz rate.



Cameras must include a string ``, which sets defaults for the frame and topic names.

By default, the ROS 2 wrapper assumes the camera is attached to a frame named `_frame`.

Additionally camera_info, color, and depth images will be published to topics called `/camera_info`, `/color`, and `/depth`, respectively.

Also note that MuJuCo's conventions for cameras are different than ROS's, and which must be accounted for.

Refer to the documentation for more information.



For example,



```xml



```



Will publish the following topics:



```bash

$ ros2 topic info /wrist_mounted_camera/camera_info

Type: sensor_msgs/msg/CameraInfo

$ ros2 topic info /wrist_mounted_camera/color

Type: sensor_msgs/msg/Image

$ ros2 topic info /wrist_mounted_camera/depth

Type: sensor_msgs/msg/Image

```



The frame and topic names are also configurable from the ros2_control xacro.

Default parameters can be overridden with:



```xml

  

  

    wrist_mounted_camera_mujoco_frame

    /wrist_mounted_camera/color/camera_info

    /wrist_mounted_camera/color/image_raw

    /wrist_mounted_camera/aligned_depth_to_color/image_raw

  

```



### Lidar



MuJoCo does not include native support for lidar sensors.

However, this package offers a ROS 2-like lidar implementation by wrapping sets of [rangefinders](https://mujoco.readthedocs.io/en/stable/XMLreference.html#sensor-rangefinder) together.



MuJoCo rangefinders measure the distance to the nearest surface along the positive `Z` axis of the sensor site.

The ROS 2 lidar wrapper uses the standard defined in [LaserScan](https://github.com/ros2/common_interfaces/blob/rolling/sensor_msgs/msg/LaserScan.msg#L10) messages.

In particular, the first rangefinder's `Z` axis (e.g. `rf-00`) must align with the ROS 2 lidar sensor's positive `X` axis.



In the MJCF, use the `replicate` tag along with a `-` separator to add N sites to attach sensors to.

For example, the following will add 12 sites named `rf-00` to `rf-11` each at a 0.025 radian offset from each other:



```xml

  

    

  

```



Then a set of rangefinders can be attached to each site with:



```xml

  

    

    

  

```



The lidar sensor is then configurable through ROS 2 control xacro with:



```xml

    

    

    

    

    

    

      lidar_sensor_frame

      0.025

      12

      0.05

      10

      /scan

    

  

```



## Test Robot System



While examples are limited, we maintain a functional example 2-dof robot system in the [test examples](./test/test_resources/test_robot.urdf) space.

We generally recommend looking there for examples and recommended workflows.



For now, built the drivers with testing enabled, then the test robot system can be launched with:



```bash

# Brings up the hardware drivers and mujoco interface, along with a single position controller

ros2 launch mujoco_ros2_control test_robot.launch.py



# Or optionally include the PID controller as mentioned above

ros2 launch mujoco_ros2_control test_robot.launch.py use_pid:=true



# Launch an rviz2 window with the provided configuration

rviz2 -d $(ros2 pkg prefix --share mujoco_ros2_control)/config/test_robot.rviz

```



From there, command joints to move with,



```bash

ros2 topic pub /position_controller/commands std_msgs/msg/Float64MultiArray "data: [-0.25, 0.75]" --once

```



> [!TIP]

> UI panels can be toggled with `Tab` or `Shift+Tab`.

> All standard MuJoCo keyboard shortcuts are available.

> To see a short list, press `F1`.



## Development



More information is provided in the [developers guide](./docs/DEVELOPMENT.md) document.