Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/neuro-mechatronics-interfaces/ros2_data_agent
Code for a multipurpose file explorer specializing in reading ROS2 topic data from '.bag' or '.db3' files
https://github.com/neuro-mechatronics-interfaces/ros2_data_agent
data python ros2
Last synced: 12 days ago
JSON representation
Code for a multipurpose file explorer specializing in reading ROS2 topic data from '.bag' or '.db3' files
- Host: GitHub
- URL: https://github.com/neuro-mechatronics-interfaces/ros2_data_agent
- Owner: Neuro-Mechatronics-Interfaces
- License: mit
- Created: 2022-06-24T17:40:59.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-01T01:11:02.000Z (11 months ago)
- Last Synced: 2024-03-01T02:28:56.638Z (11 months ago)
- Topics: data, python, ros2
- Language: Python
- Homepage:
- Size: 202 KB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# A set of python processing tools for ROS2 task experiments
---
This repository contains a set of tools designed to read bag files generated by [ROS2](https://docs.ros.org/en/iron/Installation.html) and high-level from experiment recordings. Bag files, the ROS2 standard for saving data, can be saved as '.bag' or '.db3' files and can be converted in a .mat/.txt/.csv file for further analysis. It expands upon the capabilities of the [nml_bag](https://github.com/ricmua/nml_bag) package from [Andrew Whitmore](https://github.com/ricmua) for reading bag files created from ROS2.
#### (Note: Tested in Ubuntu 20.04-22.04, and Windows 10-11 with ROS2 Galactic-Iron)
# Prerequisites
---
- [ROS2](https://docs.ros.org/en/iron/Installation.html)
- Python 3.8, with packages pandas, numpy, openpyxl (Use the package manager [pip](https://pip.pypa.io/en/stable/) to install).
# Installation
---
Start by installing the minimum library dependencies then cloning the repository:
```bash
pip install pandas numpy openpyxl bag
git clone https://github.com/Neuro-Mechatronics-Interfaces/ROS2_Data_Agent.git
```
The [nml_bag](https://github.com/ricmua/nml_bag) package can be cloned inside the `ROS2_Data_Agent` repo and run the local 'setup.py' installation file.
```bash
cd ROS2_Data_Agent
git clone https://github.com/ricmua/nml_bag.git
cd nml_bag
pip install .
```
# Quickstart/Usage
---
### Basic usage of the package follows this pattern:
1. Import package.
```python
import data_agent as da
```
2. Create the DataAgent object. A full list of potential initialization parameters (like the subject name, data search path, etc.) can be found in its [documentation](/doc/markdown/data_agent.md).
```python
agent = da.DataAgent()
```
3. Read the bag file
```python
agent.read_bag('/path/to/bagfile.mcap')
```
If there are multiple bag files to be processed, we can search for bag files in the specified folder and return a Python `dict` datatype containing a key `files` with a list of the bag file directories.
```python
file_list = agent.search_for_files('/path/to/bag/files', file_type='.mcap')
agent.read_bag(file_list[0]['files'])
```
If only bag files created on a specific date are to be read, the date can be passed as a string for the `date_tag` parameter. The date string should be in the format `DD/MM/YYYY`, and it assumes that the matching date is located in either the filename or the folder.
```python
agent.search_path = '/path/to/bag/files' # Setting the search path avoids passing it as a parameter
file_list = agent.search_for_files(file_type='.mcap', date_tag=date)
agent.read_bag(file_list[0]['files'])
```
We can also set the `combine` parameter to `True` to combine all the topic data from the bag files into continuous datasets (useful for bag files from segmented recordings for maintaining small files sizes).
```python
file_list = agent.search_for_files(file_type='.mcap', date_tag=date)
agent.read_bag(['/path/to/bag_1-of-2.mcap','/path/to/bag_2-of-2.mcap'], combine=True)
```
4. Extract topic data from the bag file and return a DataFrame object
```python
topic_df = agent.get_topic_data('/topic_name')
```
5. Save the bag file as a `.csv` file for future analysis. The `.h5`, `.hdf5`, `.txt`, and `.mat` file formats are also supported
```python
agent.save_data('/path/to/save/csv_file', file_type='.csv')
```
### TO-DO Notes:
- An `ArgParser` object is being developed to allow for command-line input for the tools. Future work will include expanding its documentation and usage
- Saving task metrics and evaluating pulse timing are being developed and will be added to the documentation soon.
- Data saving directly to a csv is supported, and documentation will be updated to include this feature soon.
# Examples
---
### A handful of example scripts are provided to demonstrate the use of the tools:
1. `example_rosbag_to_mat.py` Simple example showing how to convert a bag file and save it to a local directory as a .mat file (for MATLAB use)
2. `rosbag_to_mat.py` Converts '.bag' files to '.mat' files with terminal input support (useful for batch processing).
3. `example_save_task_metrics.py` Run file with string containing root directory as 1st argument, file type string as the second argument, and data field string as the third
4. `example_evaluate_pulse_timing.py` Grabs sync trigger and robot position data to show sampling consistency
5. `example_read_config.py` - Example of reading parameters from a configuration file