{"id":20420368,"url":"https://github.com/wkaisertexas/ros2_tracing_cpp","last_synced_at":"2025-10-09T12:10:10.059Z","repository":{"id":261130064,"uuid":"882969584","full_name":"wkaisertexas/ros2_tracing_cpp","owner":"wkaisertexas","description":"A C++ babeltrace2 plugin which replaces ROS 2 Tracing","archived":false,"fork":false,"pushed_at":"2025-03-01T03:13:07.000Z","size":810,"stargazers_count":18,"open_issues_count":3,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-12T18:09:43.095Z","etag":null,"topics":["cpp","lttng","ros2"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wkaisertexas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-11-04T06:21:38.000Z","updated_at":"2025-03-01T03:13:10.000Z","dependencies_parsed_at":"2025-02-20T21:20:21.492Z","dependency_job_id":"58bf24f0-291c-4859-9bf0-b989297e7f70","html_url":"https://github.com/wkaisertexas/ros2_tracing_cpp","commit_stats":null,"previous_names":["wkaisertexas/ros2_tracing_cpp"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/wkaisertexas/ros2_tracing_cpp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wkaisertexas%2Fros2_tracing_cpp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wkaisertexas%2Fros2_tracing_cpp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wkaisertexas%2Fros2_tracing_cpp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wkaisertexas%2Fros2_tracing_cpp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wkaisertexas","download_url":"https://codeload.github.com/wkaisertexas/ros2_tracing_cpp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wkaisertexas%2Fros2_tracing_cpp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279001439,"owners_count":26083078,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","lttng","ros2"],"created_at":"2024-11-15T06:42:35.348Z","updated_at":"2025-10-09T12:10:10.027Z","avatar_url":"https://github.com/wkaisertexas.png","language":"Jupyter Notebook","readme":"# ROS 2 Tracing C++\n\nA custom plugin for babeltrace2 which replaces [ROS 2 Tracing](https://github.com/ros2/ros2_tracing) for analyzing C++ ROS 2 nodes.\n\n\u003cp align=\"center\"\u003e\n        \u003cimg src=\"./scripts/imgs/callback_duration.png\" alt=\"Callback duration created by ROS 2 Tracing C++\" width=\"600\"/\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n        Callback execution time plot generated using ROS 2 Tracing C++\n\u003c/p\u003e\n\n## Table of Contents\n\n- [Introduction](#introduction)\n- [Alternatives to ROS 2 Tracing C++](#alternatives-to-ros-2-tracing-c)\n- [Supported Analysis Sinks](#supported-analysis-sinks)\n- [Running ROS 2 Tracing C++](#running-ros-2-tracing-c)\n- [Analyzing Processed Traces in a Jupyter Notebook](#analyzing-processed-traces-in-a-jupyter-notebook)\n- [Installation Guide](#installation-guide)\n  - [LTTNG Installation](#lttng-and-babeltrace-2-installation)\n  - [ROS 2 TraceTools Launch Installation](#ros-2-tracetools-launch-installation)\n  - [ROS 2 Tracing C++ Installation](#ros-2-tracing-c-installation)\n- [Building from Source](#building-from-source)\n  - [Babeltrace 2 from Source](#babeltrace-2-from-source)\n  - [ROS 2 Tracing C++ from Source](#ros-2-tracing-c-from-source)\n- [Collecting Traces with ROS 2 TraceTools Launch](#collecting-traces-with-ros-2-tracetools-launch)\n- [Sink Output Data Schema](#sink-output-data-schema)\n  - [Callback Duration Products](#callback-duration-products)\n  - [Memory Usage Products](#memory-usage-products)\n- [Improving Trace Processing Time](#improving-trace-processing-time)\n- [Collecting Traces in Docker](#collecting-traces-in-docker)\n- [Resources](#resources)\n\n## Introduction\n\nTrace analysis is incredibly powerful. However, processing traces with ROS 2 tracing took quite a bit longer than program execution times themselves. As a result, I wrote a custom C++ babeltrace2 plugin which uses the same mechanism of analyzing tracing but with much greater performance. As a rule of thumb, speedups of between **25x and 50x** can be expected. This enables real-time trace processing. \n\n\u003cp align=\"center\"\u003e\n        \u003cimg src=\"https://github.com/user-attachments/assets/ea88a38f-8930-40da-95a4-98e1a7485696\" alt=\"Speed comparison between ros 2 tracing and ros 2 tracing c++\" width=\"300\" /\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n        Processing times of 2 minutes of real-time trace collection on the Cavalier Autonomous Racing stack. A speedup of 52.1x and 20.9x is achieved for the memory usage and callback duration calculator, respectively.\n\u003c/p\u003e\n\n## Alternatives to ROS 2 Tracing C++\n\nThere exist other trace analysis solutions for ROS 2\n\n- [ROS 2 tracing analysis](https://github.com/ros-tracing/tracetools_analysis/tree/humble) is a good choice if you do not need high-performance trace analysis.\n- [LTTNG Analyses](https://github.com/lttng/lttng-analyses) have good examples of trace-analysis scripts written in Python if you are looking to learn more about what you can get from trace-analysis.\n\n\u003e [!IMPORTANT]\n\u003e ROS 2 Tracing and the associated tooling is quite good. Consider analyzing traces with that first as it is quick to get started. If your traces process in a reasonable amount of time, you likely do not need to use this package.\n\n## Supported Analysis Sinks\n\nThe supported features are inspired by the [ROS 2 tracing analysis](https://github.com/ros-tracing/tracetools_analysis/tree/humble) package.\n\n| Plugin | Required Tracepoints | Description |\n| :----- | :------------------- | :---------- |\n| `sink.ros2_tracing_cpp.callback_duration` | `ros2:rclcpp_callback_register`, `ros2:callback_start`, `ros2:callback_end` | Collects the duration of each callback triggered |\n| `sink.ros2_tracing_cpp.memory_usage` | `lttng_ust_libc:malloc`, `lttng_ust_libc:calloc`, `lttng_ust_libc:realloc`, `lttng_ust_libc:memalign`, `lttng_ust_libc:posix_memalign`, `lttng_ust_libc:free` | Gets information about the lifecycle of objects allocated by nodes |\n\n## Running ROS 2 Tracing C++\n\nTo run the plugin, use `babeltrace2` with the specified plugin path and your tracing session output path.\n\n```bash\nbabeltrace2 --plugin-path . ~/.ros/tracing/ros2_tracing_session/ --component=sink.ros2_tracing_cpp.memory_usage\n```\n\nAfter the trace has been fully processed, a metadata file will be produced for the run and a set of CSV files containing the processed traces. For more information about the format of produced files, see the [Sink Output Data Schema](#sink-output-data-schema) section.\n\n## Analyzing Processed Traces in a Jupyter Notebook\n\nFor each sink, there is a pre-built Jupyter notebook which makes a standard set of plots per node. These analyses are designed to be extensible and a good starting point for any future work.\n\n| Plugin | Jupyter Notebook |\n| :----- | :--------------- |\n| `sink.ros2_tracing_cpp.callback_duration` | [process_callback_duration.ipynb](./scripts/process_callback_duration.ipynb) |\n| `sink.ros2_tracing_cpp.memory_usage` | [process_callback_duration.ipynb](./scripts/process_memory_usage.ipynb) |\n\n\n## Installation Guide\n\nTo use ROS 2 Tracing C++, you must first:\n\n1. install [Linux Trace Toolkit: next generation (LTTNG)](https://lttng.org/docs/) and [Babeltrace 2](https://babeltrace.org/#bt2-get)\n2. install ROS 2 Tracetools Launch and rebuild your code\n3. download and move the plugin from the releases page\n\n### LTTNG and Babeltrace 2 Installation\n\nInstall LTTNG and Babeltrace 2 from [the stable Linux PPA package](lttng.org/docs/v2.13/#doc-ubuntu-ppa):\n\n```bash\nsudo apt-add-repository ppa:lttng/stable-2.13\nsudo apt-get update\nsudo apt install -y lttng-tools lttng-modules-dkms liblttng-ust-dev\nsudo apt install -y babeltrace2\n```\n\n### ROS 2 TraceTools Launch Installation\n\nInstalling tracetools requires `apt` installing the `tracetools-launch` package and then sourcing the tracetools package **before** building any of the other packages which should be instrumented.\n\n```bash\nsudo apt install ros-humble-tracetools-launch\n(cd external \u0026\u0026 git clone https://github.com/ros2/ros2_tracing \u0026\u0026 cd ros2_tracing \u0026\u0026 git checkout humble)\ncolcon build --packages-up-to tracetools\nsource install/setup.sh\n```\n\nAfter this, you can build your packages with `colcon` like normal. \n\n\u003e [!WARNING]\n\u003e However, you must `source install/setup.sh` before building any of the other packages you want tracepoints enabled in. This is **atypical** for a ROS 2 project but is a valid workaround to building with tracepoints enabled while not requiring a from-source ROS build.\n\n### ROS 2 Tracing C++ Installation\n\nIf you are running on a x86 system, you can [download the latest release binary](https://github.com/wkaisertexas/ros2_tracing_cpp/releases/latest) from the GitHub release.\n\n```bash\nwget https://github.com/wkaisertexas/ros2_tracing_cpp/releases/download/v1/libros2_tracing_cpp.so\nmkdir -p /usr/local/lib/babeltrace2/plugins\nsudo cp libros2_tracing_cpp.so /usr/local/lib/babeltrace2/plugins\n```\n\n\u003e [!NOTE]\n\u003e Moving `libros2_tracing_cpp.so` to `/usr/local/lib/babeltrace2/plugins` is optional but allows the trace analysis sinks to be used without specifying `--plugin-path`.\n\n## Building from Source\n\nBuilding ROS 2 Tracing C++ from source requires:\n\n1. build `babeltrace2` from source\n2. build the plugin `libros2_tracing_cpp.so`\n\n### Babeltrace 2 from Source\n\nTo build a plugin, building from source is required ([see guide](https://babeltrace.org/docs/v2.0/libbabeltrace2/guide-build-bt2-dev.html)). This assumes you have a `.gitignore`-d directory named `external` for non-source packages.\n\n```bash\n# downloading and extracting babeltrace2\nmkdir -p external\ncd external\ncurl https://www.efficios.com/files/babeltrace/babeltrace2-2.0.6.tar.bz2 -o babeltrace.tar.bz2\ntar -xvf babeltrace.tar.bz2\ncd babeltrace2-*/\n\n# building an installing from source\nBABELTRACE_DEV_MODE=1 BABELTRACE_MINIMAL_LOG_LEVEL=TRACE ./configure --disable-debug-info\nmake -j$(nproc)\nsudo make install\ncd ../..\n```\n\nAfter that, `babeltrace2` will be built from source and installed.\n\n### ROS 2 Tracing C++ from Source\n\nTo build the ROS 2 Tracing C++ plugin from source, run:\n\n```bash\ngit clone https://github.com/wkaisertexas/ros2_tracing_cpp\ncd ros2_tracing_cpp\n\nmkdir build\ncd build\ncmake ..\nmake -j$(nproc)\nsudo make install\n```\n\nAt this point, the plugin will be built in `build/plugins/libros2_tracing_cpp.so` where you can reference the different sinks to process your traces.\n\n## Collecting Traces with ROS 2 TraceTools Launch\n\nWith `tracetools-launch` installed, you can add the following to your launch file to collect traces.\n\n```python\nfrom launch import LaunchDescription\n\nfrom tracetools_launch.action import Trace\n\nCALLBACK_TRACEPOINTS = set([\n    \"ros2:rclcpp_callback_register\",\n    \"ros2:callback_start\",\n    \"ros2:callback_end\",\n])\n\"\"\"Tracepoints required for callback duration analysis\"\"\"\n\nMEMORY_TRACEPOINTS = set([\n    \"lttng_ust_libc:malloc\",\n    \"lttng_ust_libc:calloc\",\n    \"lttng_ust_libc:realloc\",\n    \"lttng_ust_libc:memalign\",\n    \"lttng_ust_libc:posix_memalign\",\n    \"lttng_ust_libc:free\"\n])\n\"\"\"Tracepoints required for memory usage analysis\"\"\"\n\ndef generate_launch_description() -\u003e LaunchDescription:\n    tracepoints_to_collect = CALLBACK_TRACEPOINTS | MEMORY_TRACEPOINTS\n    trace_session = Trace(\n        session_name=\"callback_duration_and_memory_usage\",\n        events_ust=list(tracepoints_to_collect),\n        base_path=\"~/.ros/tracing\", # default trace location\n    )\n\n    # the rest of your launch file\n\n    return LaunchDescription([\n        trace_session,\n        # your nodes and parameters\n    ])\n```\n\nThis launch command will create `~/.ros/tracing/callback_duration_and_memory_usage` which contains several nested folders. Running `babeltrace2 ~/.ros/tracing/callback_duration_and_memory_usage | less` will print out traces.\n\n## Sink Output Data Schema\n\nIn this section, notes and helpful tips to process outputs generated by each plugin is included.\n\n### Callback Duration Products\n\nFor files produced by `callback_duration`, a `callback_duration_metadata.csv` file which contains links and metadata to `callback_*.csv` files is produced. A script called [`scripts/process_callback_duration.ipynb`](./scripts/process_callback_duration.ipynb) makes callback duration plots using the collected traces.\n\n| Column | Description |\n| :----- | :---------- |\n| `symbol` | The raw callback symbol monitored |\n| `procname` | Process name of collected callback | \n| `address`  | The address of the selected callback, used as a pseudo-identifier |\n| `count` | the number of times the callback was called |\n| `path` | the relative path of the file containing the callback |\n| `avg_duration` | the average duration of the callback (useful for crude filtering) |\n\nThe file contained in the path variable is a CSV file. The first two rows are the `symbol` and the `procname`. Then the file has the following columns:\n\n| Column | Description |\n| :----- | :---------- |\n| `time`   | time in nanoseconds since the unix epoch that the callback occurred |\n| `duration` | the duration of the callback in nanoseconds |\n\n\u003e [!TIP]\n\u003e If you are processing this in Pandas, you can use `pd.read_csv(path, skiprows=2)` to ignore the header\n\n### Memory Usage Products\n\nFiles produced by `memory_usage`, you get a `memory_usage_metadata.csv` file which contains links and metadata to `mem_*.csv` files. A script called [`scripts/process_memory_usage.ipynb`](./scripts/process_memory_usage.ipynb) makes a 2x2 grid of plots examining allocations over time and the relative frequency of allocations of differing sizes.\n\n| Columns | Description |\n| :------ | :---------- |\n| `path`    | the relative path to the `.csv` file containing the allocation information |\n| `vpid`    | the virtual pid of the ros2 node |\n| `procname` | process name making the allocation |\n| `avg_alloc_lifecycle` | the average lifecycle of allocated objects |\n| `avg_alloc_size` | the average allocation size |\n| `allocation_count` | the total number of allocations |\n| `max_process_memory` | the maximum amount of process memory |\n\nThe file contained in the `path` variable is a CSV file containing the following information:\n\n| Columns | Description |\n| :------ | :---------- |\n| `type` | type of the allocation either `malloc`, `calloc`, `realloc`, `memalign` or `posix_memalign` |\n| `time` | time in nanoseconds since the unix epoch the object was allocated |\n| `duration` | how long the memory stuck around. A nan value means that the object was not freed |\n| `prev_size` | the previous allocation size in bytes (only for realloc) |\n| `size` | the size of the allocation in bytes |\n| `vtid` | the virtual thread id which called the allocator | \n\n## Improving Trace Processing Time\n\nLooking at a profile of this plugin, iterating through the traces with babeltrace2 takes **over 85% of total time**. Collecting the minimal set of events required for each plugin is the single-greatest tactic for reducing program execution time.\n\n![Perf Trace of `memory_usage`](./scripts/imgs/perf_data.png)\n\n\u003e [!IMPORTANT]\n\u003e To speed up processing time, consider using a live session created with `llttng create my-session --live` and `babeltrace2 --plugin-path . --input-format=lttng-live net://localhost/host/localhost/my-session --component=sink.ros2_tracing_cpp.memory_usage`\n\n## Collecting Traces in Docker\n\nCollecting traces inside a Docker container requires a non-standard setup. Namely, you **do not** install `lttng-modules-dkms`\n\n```dockerfile\nRUN apt update \u0026\u0026 apt install -y ros-humble-tracetools-* babeltrace2 lttng-tools liblttng-ust-dev\n```\n\nAfter this, running your docker image with the `--privileged` flag will allow the collection of both userspace and kernel traces inside the container.\n\n## Resources\n\n1. [Babeltrace 2 Sink Example](https://babeltrace.org/docs/v2.0/libbabeltrace2/example-simple-sink-cmp-cls.html)\n2. [ROS 2 Trace Analysis](https://github.com/ros-tracing/tracetools_analysis/tree/humble)\n3. [LTTNG Documentation](https://lttng.org/docs/v2.13/)\n4. [ROS 2 Tracing White Paper](https://arxiv.org/abs/2201.00393)\n5. [ROS 2 Tracing Overhead Evaluation](https://github.com/christophebedard/ros2_tracing-overhead-evaluation)\n6. [ROS 2 Tracing GitHub](https://github.com/ros2/ros2_tracing/tree/humble)\n\n\u003e [!NOTE]\n\u003e If you liked this repository, please consider giving it a star!\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwkaisertexas%2Fros2_tracing_cpp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwkaisertexas%2Fros2_tracing_cpp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwkaisertexas%2Fros2_tracing_cpp/lists"}