{"id":24030340,"url":"https://github.com/neuro-mechatronics-interfaces/ros2_data_agent","last_synced_at":"2026-06-13T12:05:48.443Z","repository":{"id":39623151,"uuid":"507095912","full_name":"Neuro-Mechatronics-Interfaces/ROS2_Data_Agent","owner":"Neuro-Mechatronics-Interfaces","description":"Code for a multipurpose file explorer specializing in reading ROS2 topic data from '.bag' or '.db3' files","archived":false,"fork":false,"pushed_at":"2024-03-01T01:11:02.000Z","size":207,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-26T03:44:58.214Z","etag":null,"topics":["data","python","ros2"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Neuro-Mechatronics-Interfaces.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-06-24T17:40:59.000Z","updated_at":"2023-04-01T15:21:49.000Z","dependencies_parsed_at":"2024-02-15T22:50:22.389Z","dependency_job_id":null,"html_url":"https://github.com/Neuro-Mechatronics-Interfaces/ROS2_Data_Agent","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Neuro-Mechatronics-Interfaces/ROS2_Data_Agent","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neuro-Mechatronics-Interfaces%2FROS2_Data_Agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neuro-Mechatronics-Interfaces%2FROS2_Data_Agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neuro-Mechatronics-Interfaces%2FROS2_Data_Agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neuro-Mechatronics-Interfaces%2FROS2_Data_Agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Neuro-Mechatronics-Interfaces","download_url":"https://codeload.github.com/Neuro-Mechatronics-Interfaces/ROS2_Data_Agent/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neuro-Mechatronics-Interfaces%2FROS2_Data_Agent/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34283442,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-13T02:00:06.617Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","python","ros2"],"created_at":"2025-01-08T17:42:08.451Z","updated_at":"2026-06-13T12:05:48.421Z","avatar_url":"https://github.com/Neuro-Mechatronics-Interfaces.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# A set of python processing tools for ROS2 task experiments\n\n---\nThis repository contains a set of tools designed to read bag files generated by [ROS2](https://docs.ros.org/en/iron/Installation.html) and high-level from experiment recordings. Bag files, the ROS2 standard for saving data, can be saved as '.bag' or '.db3' files and can be converted in a .mat/.txt/.csv file for further analysis. It expands upon the capabilities of the  [nml_bag](https://github.com/ricmua/nml_bag) package from [Andrew Whitmore](https://github.com/ricmua) for reading bag files created from ROS2.\n\n#### (Note: Tested in Ubuntu 20.04-22.04, and Windows 10-11 with ROS2 Galactic-Iron)\n\n# Prerequisites\n\n---\n- [ROS2](https://docs.ros.org/en/iron/Installation.html)\n- Python 3.8, with packages pandas, numpy, openpyxl (Use the package manager [pip](https://pip.pypa.io/en/stable/) to install).\n\n# Installation\n\n---\nStart by installing the minimum library dependencies then cloning the repository:\n```bash\npip install pandas numpy openpyxl bag\ngit clone https://github.com/Neuro-Mechatronics-Interfaces/ROS2_Data_Agent.git\n```\nThe [nml_bag](https://github.com/ricmua/nml_bag) package can be cloned inside the `ROS2_Data_Agent` repo and run the local 'setup.py' installation file.\n```bash\ncd ROS2_Data_Agent\ngit clone https://github.com/ricmua/nml_bag.git\ncd nml_bag\npip install .\n   ```\n# Quickstart/Usage\n\n--- \n### Basic usage of the package follows this pattern:\n\n1. Import package.\n   ```python\n   import data_agent as da\n   ```\n2. Create the DataAgent object. A full list of potential initialization parameters (like the subject name, data search path, etc.) can be found in its [documentation](/doc/markdown/data_agent.md).\n   ```python \n   agent = da.DataAgent()\n   ```\n   \n\n3. Read the bag file\n    ```python\n   agent.read_bag('/path/to/bagfile.mcap')\n   ```\n   If there are multiple bag files to be processed, we can search for bag files in the specified folder and return a Python `dict` datatype containing a key `files` with a list of the bag file directories.\n   ```python\n   file_list = agent.search_for_files('/path/to/bag/files', file_type='.mcap')\n   agent.read_bag(file_list[0]['files'])\n   ```   \n   If only bag files created on a specific date are to be read, the date can be passed as a string for the `date_tag` parameter. The date string should be in the format `DD/MM/YYYY`, and it assumes that the matching date is located in either the filename or the folder.\n   ```python\n   agent.search_path = '/path/to/bag/files' # Setting the search path avoids passing it as a parameter\n   file_list = agent.search_for_files(file_type='.mcap', date_tag=date)\n   agent.read_bag(file_list[0]['files'])\n   ``` \n   We can also set the `combine` parameter to `True` to combine all the topic data from the bag files into continuous datasets (useful for bag files from segmented recordings for maintaining small files sizes).\n   ```python\n   file_list = agent.search_for_files(file_type='.mcap', date_tag=date)\n   agent.read_bag(['/path/to/bag_1-of-2.mcap','/path/to/bag_2-of-2.mcap'], combine=True)\n   ```\n4. Extract topic data from the bag file and return a DataFrame object\n   ```python\n   topic_df = agent.get_topic_data('/topic_name')\n   ```   \n5. Save the bag file as a `.csv` file for future analysis. The `.h5`, `.hdf5`, `.txt`, and `.mat` file formats are also supported\n   ```python\n   agent.save_data('/path/to/save/csv_file', file_type='.csv')\n   ```\n   \n### TO-DO Notes: \n- An `ArgParser` object is being developed to allow for command-line input for the tools. Future work will include expanding its documentation and usage\n- Saving task metrics and evaluating pulse timing are being developed and will be added to the documentation soon.\n- Data saving directly to a csv is supported, and documentation will be updated to include this feature soon.\n\n# Examples\n\n---\n### A handful of example scripts are provided to demonstrate the use of the tools:\n1. `example_rosbag_to_mat.py` Simple example showing how to convert a bag file and save it to a local directory as a .mat file (for MATLAB use)\n2. `rosbag_to_mat.py` Converts '.bag' files to '.mat' files with terminal input support (useful for batch processing).\n3. `example_save_task_metrics.py` Run file with string containing root directory as 1st argument, file type string as the second argument, and data field string as the third\n4. `example_evaluate_pulse_timing.py` Grabs sync trigger and robot position data to show sampling consistency\n5. `example_read_config.py` - Example of reading parameters from a configuration file\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneuro-mechatronics-interfaces%2Fros2_data_agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fneuro-mechatronics-interfaces%2Fros2_data_agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneuro-mechatronics-interfaces%2Fros2_data_agent/lists"}