https://github.com/sslab-gatech/robofuzz

Fuzzing framework for Robot Operating System (ROS) and ROS-based robotic systems
https://github.com/sslab-gatech/robofuzz

Last synced: 9 months ago
JSON representation

Fuzzing framework for Robot Operating System (ROS) and ROS-based robotic systems

Host: GitHub
URL: https://github.com/sslab-gatech/robofuzz
Owner: sslab-gatech
License: mit
Created: 2022-08-09T01:50:41.000Z (over 3 years ago)
Default Branch: master
Last Pushed: 2023-02-15T20:03:12.000Z (almost 3 years ago)
Last Synced: 2025-03-21T15:45:23.256Z (10 months ago)
Language: Python
Homepage: https://2022.esec-fse.org/details/fse-2022-research-papers/86/RoboFuzz-Fuzzing-Robotic-Systems-over-Robot-Operating-System-ROS-for-Finding-Corre
Size: 206 KB
Stars: 19
Watchers: 25
Forks: 8
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

# RoboFuzz

RoboFuzz is a fuzzing framework for testing Robot Operating System 2 (ROS 2),
and robotic systems that are built using ROS 2. Any developer-defined
properties relating to the correctness of the robotic system under test,
e.g., conformance to specification, can be tested using RoboFuzz.

For technical details,
please check our [paper](https://squizz617.github.io/pubs/robofuzz-fse22.pdf),
"RoboFuzz: Fuzzing Robotic Systems over Robot Operating System (ROS)
for Finding Correctness Bugs", which was published in ESEC/FSE 2022.

We tested six targets with RoboFuzz;

* Two from the internal layers of ROS2 foxy:
1. Type system (ROSIDL)
2. ROS Client Library APIs (rclpy and rclcpp)

* Four ROS-based robotic systems/libraries:

3. Turtlesim (apt pkg: ros-foxy-turtlesim)
4. Move It 2 + PANDA manipulator (included in moveit2_tutorials)
5. TurtleBot3 Burger (ver. foxy)
6. PX4 quadcopter (firmware v1.12 + fmu-v5)

We built test oracles by studying and encoding the correctness semantics
of each target. These oracles are capable of detecting three types of semantic
correctness bugs:
1. Violation of physical laws
2. Violation of specification
3. Cyber-physical discrepancy

## Getting RoboFuzz

Before you start, please refer to `REQUIREMENTS.md` for hardware and software
requirements, and check `INSTALL.md` for the instructions on the installation
and basic usage.

In a nutshell, you can obtain and use RoboFuzz by getting the docker image:
```
$ docker pull ghcr.io/sslab-gatech/robofuzz:latest
$ docker tag ghcr.io/sslab-gatech/robofuzz:latest robofuzz
```
(Due to the pre-compiled target robotic systems, the size of the image is
approximately 20 GB, so docker pull might take a while.)

The docker image includes the code of RoboFuzz, along with the pre-compiled
packages of the six targets and all the tools required for experiments:
* `/robofuzz/src/`: Source code of RoboFuzz
* `/robofuzz/src/fuzzer.py`: Main module of RoboFuzz
* `/robofuzz/src/oracles/*`: Correctness oracles for target systems
* `/robofuzz/src/watchlist/*`: Topics to monitor
* `/robofuzz/targets`: Pre-compiled test targets
* `/robofuzz/ros2_foxy`: ROS2 foxy installation (instrumented ver.)
* `/opt/ros/foxy`: ROS2 foxy installation (package manager ver.)

## Running RoboFuzz and reproducing our results

RoboFuzz needs to be run separately for each target system.

### 0. Understanding the output of RoboFuzz

RoboFuzz creates an output directory to log all runtime information and
metadata. Unless the path is explicitly specified through `--logdir LOGDIR`,
you can find the logs under `/robofuzz/src/logs/timestamp` in the container.
In addition, `/robofuzz/src/logs/latest` always points (i.e., symlinks) to the
latest log directory.

* Glossary
* Exec: One exec of fuzzing includes running the target system, publishing
a mutated message, checking for errors, and terminating the target.
* Round: A round of fuzzing consists of multiple executions under a specific
mutation schedule. Check `/robofuzz/src/scheduler.py` for details.
* Cycle: A cycle is a collection of several rounds. The number of rounds in
a cycle is configurable. In general, RoboFuzz keeps mutating one
message during one cycle, and moves on to the next message in the
queue when the new cycle begins.
* Frame: The timestamp taken at the beginning of a round.
* Subframe: The timestamp taken when each message is published. For example,
if an execution begins at `ts-0`, it is the frame. If, during
the execution, RoboFuzz publishes a sequence of three messages
at `ts-1`, `ts-2`, and, `ts-3` respectively, each timestamp is
the subframe of the corresponding message. Using the combination
of the frame and subframe, any message can be located.

* Each log directory has several sub-directories:
* `metadata/`: contains node, topic, data type, cycle and round count of
each execution.
* Filename format: `meta-{Frame}`
* `errors/`: Error messages emitted by oracles when bugs are triggered.
* Filename format: `error-{Frame}`.
* `cov/`: Feedback elements and their values when they are considered
interesting.
* Filename format: `{Frame}`.
* `queue/`: Messages generated/mutated during fuzzing. If multiple messages
were published during one execution, e.g., with `SEQUENCE`
schedule, they can be retrieved by `ls queue | grep {Frame}`.
* Filename format: `msg-{Frame}-{Subframe}`.
* `rosbags/`: Rosbag files (i.e., dump of all messages and states) of buggy
test cases. These are standard rosbags of ROS 2, and for
detailed usage, please refer to the [official tutorial](https://docs.ros.org/en/foxy/Tutorials/Beginner-CLI-Tools/Recording-And-Playing-Back-Data/Recording-And-Playing-Back-Data.html).
* Filename format: `{Frame}/states-{exec}.bag`

During or after the fuzzing campaign, you can refer to the contents of the log
directory to make sense of the fuzzing progress and detected bugs.

### 1. Testing ROSIDL