{"id":50281100,"url":"https://github.com/picknikrobotics/data_tamer","last_synced_at":"2026-05-28T00:05:19.724Z","repository":{"id":205098924,"uuid":"713299308","full_name":"PickNikRobotics/data_tamer","owner":"PickNikRobotics","description":"C++ library for Fearless Timeseries Logging","archived":false,"fork":false,"pushed_at":"2025-10-07T19:59:20.000Z","size":1640,"stargazers_count":271,"open_issues_count":12,"forks_count":25,"subscribers_count":17,"default_branch":"main","last_synced_at":"2025-10-07T21:31:06.669Z","etag":null,"topics":["logging","plotjuggler","real-time","timeseries-data","tracing"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PickNikRobotics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-11-02T08:40:51.000Z","updated_at":"2025-10-04T22:01:30.000Z","dependencies_parsed_at":"2023-12-14T20:57:47.858Z","dependency_job_id":"cbbf400c-6dc0-4a30-81cb-1f47c5dcda22","html_url":"https://github.com/PickNikRobotics/data_tamer","commit_stats":{"total_commits":190,"total_committers":4,"mean_commits":47.5,"dds":"0.015789473684210575","last_synced_commit":"8cd4e05a804850b6c02945bd0617405692bde9b2"},"previous_names":["facontidavide/data_tamer","picknikrobotics/data_tamer"],"tags_count":22,"template":false,"template_full_name":null,"purl":"pkg:github/PickNikRobotics/data_tamer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PickNikRobotics%2Fdata_tamer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PickNikRobotics%2Fdata_tamer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PickNikRobotics%2Fdata_tamer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PickNikRobotics%2Fdata_tamer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PickNikRobotics","download_url":"https://codeload.github.com/PickNikRobotics/data_tamer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PickNikRobotics%2Fdata_tamer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33588370,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-27T02:00:06.184Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["logging","plotjuggler","real-time","timeseries-data","tracing"],"created_at":"2026-05-28T00:05:16.722Z","updated_at":"2026-05-28T00:05:19.711Z","avatar_url":"https://github.com/PickNikRobotics.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Data Tamer](data_tamer_logo.png)\n\n[![cmake Ubuntu](https://github.com/facontidavide/data_tamer/actions/workflows/cmake_ubuntu.yml/badge.svg)](https://github.com/facontidavide/data_tamer/actions/workflows/cmake_ubuntu.yml)\n[![ros2](https://github.com/PickNikRobotics/data_tamer/actions/workflows/ros2.yml/badge.svg)](https://github.com/PickNikRobotics/data_tamer/actions/workflows/ros2.yml)\n[![codecov](https://codecov.io/gh/facontidavide/data_tamer/graph/badge.svg?token=D0wtsntWds)](https://codecov.io/gh/facontidavide/data_tamer)\n\n**DataTamer** is a library to log/trace numerical variables over time and\ntakes periodic \"snapshots\" of their values, to later visualize them as **timeseries**.\n\nIt works great with [PlotJuggler](https://github.com/facontidavide/PlotJuggler),\nthe timeseries visualization tool (note: you will need PlotJuggler **3.8.2** or later).\n\n**DataTamer** is \"fearless data logger\" because you can record hundreds or **thousands of variables**:\neven 1 million points per second should have a fairly small CPU overhead.\n\nSince all the values are aggregated in a single \"snapshot\", it is usually meant to\nrecord data in a periodic loop (a very frequent use case, in robotics applications).\n\nKudos to [pal_statistics](https://github.com/pal-robotics/pal_statistics), for inspiring this project.\n\n## How it works\n\n![architecture](concepts.png)\n\nDataTamer can be used to monitor multiple variables in your applications.\n\n**Channels** are used to take \"snapshots\" of a subset of variables at a given time.\nIf you want to record at different frequencies, you can use different channels.\n\nDataTamer will forward the collected data to 1 or multiple **sinks**;\na sink may save the information immediately in a file (currently, we support [MCAP](https://mcap.dev/))\nor publish it using an inter-process communication, for instance, a ROS2 publisher.\n\nYou can easily create your own, specialized sinks.\n\nUse [PlotJuggler](https://github.com/facontidavide/PlotJuggler) to\nvisualize your logs offline or in real-time.\n\n## Features\n\n- **Serialization schema is created at run-time**: no need to do code generation.\n- **Suitable for real-time applications**: very low latency (on the side of the callee).\n- **Multi-sink architecture**: recorded data can be forwarded to multiple \"backends\".\n- **Very low serialization overhead**, in the order of 1 bit per traced value.\n- The user can enable/disable traced variables at run-time.\n\n## Limitations\n\n- Traced variables can not be added (registered) once the recording starts (first `takeSnapshot`).\n- Focused on periodic recording. Not the best option for sporadic, asynchronous events.\n- If you use `DataTamer::registerValue` you must be careful about the lifetime of the\nobject. If you prefer a safer RAII interface, use `DataTamer::createLoggedValue` instead.\n\n# Examples\n\n## Basic example\n\n```cpp\n#include \"data_tamer/data_tamer.hpp\"\n#include \"data_tamer/sinks/mcap_sink.hpp\"\n\nint main()\n{\n  // Multiple channels can use this sink. Data will be saved in mylog.mcap\n  auto mcap_sink = std::make_shared\u003cDataTamer::MCAPSink\u003e(\"mylog.mcap\");\n\n  // Create a channel and attach a sink. A channel can have multiple sinks\n  auto channel = DataTamer::LogChannel::create(\"my_channel\");\n  channel-\u003eaddDataSink(mcap_sink);\n\n  // You can register any arithmetic value. You are responsible for their lifetime!\n  double value_real = 3.14;\n  int value_int = 42;\n  auto id1 = channel-\u003eregisterValue(\"value_real\", \u0026value_real);\n  auto id2 = channel-\u003eregisterValue(\"value_int\", \u0026value_int);\n\n  // If you prefer to use RAII, use this method instead\n  // logged_real will unregister itself when it goes out of scope.\n  auto logged_real = channel-\u003ecreateLoggedValue\u003cfloat\u003e(\"my_real\");\n\n  // Store the current value of all the registered values\n  channel-\u003etakeSnapshot();\n\n  // You can disable (i.e., stop recording) a value like this\n  channel-\u003esetEnabled(id1, false);\n  // or, in the case of a LoggedValue\n  logged_real-\u003esetEnabled(false);\n\n  // The next snapshot will contain only [value_int], i.e. [id2],\n  // since the other two were disabled\n  channel-\u003etakeSnapshot();\n}\n```\n## How to register custom types\n\nContainers such as `std::vector` and `std::array` are supported out of the box.\nYou can also register a custom type, as shown in the example below.\n\n```cpp\n#include \"data_tamer/data_tamer.hpp\"\n#include \"data_tamer/sinks/mcap_sink.hpp\"\n#include \"data_tamer/custom_types.hpp\"\n\n// This is your custom type\nnamespace MyNamespace\n{\nstruct Point3D\n{\n  double x;\n  double y;\n  double z;\n};\n} // end namespace MyNamespace\n\n// You must implement the function TypeDefinition in the same namespace as Point3D\nnamespace MyNamespace\n{\ntemplate \u003ctypename AddField\u003e\nstd::string_view TypeDefinition(Point3D\u0026 point, AddField\u0026 add) {\n  add(\"x\", \u0026point.x);\n  add(\"y\", \u0026point.y);\n  add(\"z\", \u0026point.z);\n  return \"Point3D\";\n}\n} // end namespace MyNamespace\n\nint main()\n{\n  auto channel = DataTamer::LogChannel::create(\"my_channel\");\n  channel-\u003eaddDataSink(std::make_shared\u003cDataTamer::MCAPSink\u003e(\"mylog.mcap\"));\n\n  // Array/vectors are supported natively\n  std::vector\u003cdouble\u003e values = {1, 2, 3, 4};\n  channel-\u003eregisterValue(\"values\", \u0026values);\n\n  // Requires the implementation of DataTamer::TypeDefinition\u003cPoint3D\u003e\n  Point3D position = {0.1, -0.2, 0.3};\n  channel-\u003eregisterValue(\"position\", \u0026position);\n\n  // save the data as usual ...\n  channel-\u003etakeSnapshot();\n}\n```\n\n# Compilation\n\n## Compiling with ROS2\n\nJust use colcon :)\n\n## Compiling with Conan (not ROS2 support)\n\nNote that the ROS2 publisher will **NOT** be built when using this method.\n\nAssuming conan 2.x installed. From the source directory.\n\n**Release**:\n\n```\nconan install . -s compiler.cppstd=gnu17 --build=missing -s build_type=Release\ncmake -S . -B build/Release -DCMAKE_BUILD_TYPE=Release \\\n      -DCMAKE_TOOLCHAIN_FILE=\"build/Release/generators/conan_toolchain.cmake\"\ncmake --build build/Release --parallel\n```\n\n**Debug**:\n\n```\nconan install . -s compiler.cppstd=gnu17 --build=missing -s build_type=Debug\ncmake -S . -B build/Debug -DCMAKE_BUILD_TYPE=Debug \\\n      -DCMAKE_TOOLCHAIN_FILE=\"build/Debug/generators/conan_toolchain.cmake\"\ncmake --build build/Debug --parallel\n```\n\n# How to deserialize data recorded with DataTamer\n\nI will write more extensively about the serialization format used by DataTamer, but for the time being I\ncreated a single header file without external dependencies that you can just copy into your project:\n[data_tamer_parser.hpp](data_tamer_cpp/include/data_tamer_parser/data_tamer_parser.hpp)\n\nYou can see how it is used in this example: [mcap_reader](data_tamer_cpp/examples/mcap_reader.cpp)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpicknikrobotics%2Fdata_tamer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpicknikrobotics%2Fdata_tamer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpicknikrobotics%2Fdata_tamer/lists"}