{"id":29021678,"url":"https://github.com/mapiv/localization_evaluation_toolkit","last_synced_at":"2025-10-13T19:32:17.752Z","repository":{"id":46146120,"uuid":"492634066","full_name":"MapIV/localization_evaluation_toolkit","owner":"MapIV","description":null,"archived":false,"fork":false,"pushed_at":"2024-08-20T05:52:06.000Z","size":5177,"stargazers_count":55,"open_issues_count":1,"forks_count":8,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-08-21T07:16:56.334Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MapIV.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-16T00:10:31.000Z","updated_at":"2024-08-14T10:21:55.000Z","dependencies_parsed_at":"2024-08-20T07:14:50.799Z","dependency_job_id":"1ad5d6bf-ad22-4dfc-8889-1e18e4501df2","html_url":"https://github.com/MapIV/localization_evaluation_toolkit","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/MapIV/localization_evaluation_toolkit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MapIV%2Flocalization_evaluation_toolkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MapIV%2Flocalization_evaluation_toolkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MapIV%2Flocalization_evaluation_toolkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MapIV%2Flocalization_evaluation_toolkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MapIV","download_url":"https://codeload.github.com/MapIV/localization_evaluation_toolkit/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MapIV%2Flocalization_evaluation_toolkit/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261984645,"owners_count":23240305,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-26T02:07:27.264Z","updated_at":"2025-10-13T19:32:12.717Z","avatar_url":"https://github.com/MapIV.png","language":"Python","readme":"# localization_evaluation_toolkit\n\nAn evaluator for localization results, which compares with a reliable pose trajectory. Specifically, comparisons between different start times, end times, and periods are supported, since evaluation will be automatically aligned with the one with fewer data.\n\n## 1. Installation \u0026 Preparation\n\nClone this repo to the local environment.\n\n```sh\n$ git clone https://github.com/MapIV/localization_evaluation_toolkit.git\n```\n\nEvaluatable data types are limited to the format of CSV or ros2 bag, and the start time, end time, and period can be different for each data.\nSpecifically, a valid **CSV file** requires at least the following entries.\n\n- Timestamp\n- Position (x,y,z)\n- Rotation (Quaternion or Euler[degree or radian])\n\nAnd a valid ros2 bag file should have one of the following types of topics.\n\n- `geometry_msgs/msg/PoseWithCovarianceStamped`\n- `nav_msgs/msg/Odometry`\n\n## 2. Main-evaluation\n\n### 2.1 Pose evaluation\n\nA standard configuration file for pose evaluation consists of the following parts.\n\n- Reference\n- Result1, Result2, ...\n- Optional settings\n\nIt is required that the reference block should have the name of ***Reference*** and the names of result blocks should begin with ***Result***.\nWhen specifying the paths, it is recommended to use the absolute path.\nThere is a sample YAML file at `config/evaluation_pose.yaml`.\nTo create your own configuration file, just use the following templates for CSV or ros2 bag,\n\n\u003cdetails\u003e\n\u003csummary\u003eTemplate of CSV (click to expand)\u003c/summary\u003e\n\n```yaml\nReference/Result:\n  ## Auxiliary info\n  label: any\n  type: 0 # [0]: csv, [1]: ros2bag\n  path: /path/to/csv\n\n  ## Time\n  separate_time_stamp: false # [true]:Set secs_stamp_column and nsecs_stamp_column / [false]:Set stamp_column\n  #--------true--------#\n  secs_stamp_column: 2\n  nsecs_stamp_column: 3\n  #--------false-------#\n  stamp_column: 0\n\n  ## Position\n  x_column: 1\n  y_column: 2\n  z_column: 3\n\n  ## Rotation\n  use_quaternion: false # [true]:Set Quaternion / [false]:Set Euler\n  #--------true--------#\n  # Quaternion\n  ori_x_column: 8\n  ori_y_column: 9\n  ori_z_column: 10\n  ori_w_column: 11\n  #--------false-------#\n  # Euler\n  use_radian: true # [true]:radian / [false]:degree\n  roll_column: 4\n  pitch_column: 5\n  yaw_column: 6\n\n  ## TF\n  tf_time: 0   # [s]\n  tf_x: 0      # [m]\n  tf_y: 0      # [m]\n  tf_z: 0      # [m]\n  tf_roll: 0   # [rad]\n  tf_pitch: 0  # [rad]\n  tf_yaw: 0    # [rad]\n  inv_roll: 1  # 1 or -1\n  inv_pitch: 1 # 1 or -1\n  inv_yaw: 1   # 1 or -1\n\n  # Display ellipse (put 2D covariance in result file)\n  display_ellipse: false\n  covariance_xx_column: 10\n  covariance_xy_column: 11\n  covariance_yx_column: 12\n  covariance_yy_column: 13\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eTemplate of ros2 bag (click to expand)\u003c/summary\u003e\n\n```yaml\nReference/Result:\n  ## Auxiliary info\n  label: any\n  type: 1 # [0]: csv, [1]: ros2bag\n  path: /path/to/ros2bag\n\n  ## Rosbag info\n  topic_name: /localization/pose_estimator/pose_with_covariance\n  storage_id: sqlite3\n  serialization_format: cdr\n\n  ## TF\n  tf_time: 0   # [s]\n  tf_x: 0      # [m]\n  tf_y: 0      # [m]\n  tf_z: 0      # [m]\n  tf_roll: 0   # [rad]\n  tf_pitch: 0  # [rad]\n  tf_yaw: 0    # [rad]\n  inv_roll: 1  # 1 or -1\n  inv_pitch: 1 # 1 or -1\n  inv_yaw: 1   # 1 or -1\n\n  # Display ellipse (put 2D covariance in result file)\n  display_ellipse: false # use PoseWithCovarianceStamped tyep topic in result data\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eTemplate of optional settings (click to expand)\u003c/summary\u003e\n\n```yaml\n# Horizontal axis\naxis_type: 0   # [0]:time, [1]:distance\ndegree_type: 0 # [0]:radian, [1]:degree\n\n# Trajectory graph dilution\ndilution_step: 10 # at least 1, the larger the sparser for better performance\n\n# Trajectory graph numbering\nprogress_info: 0 # [0]:off, [1]:number, [2]:time, [3]:ros time, [4]:distance\ninterval: 0      # progress_info is [2]:second, [3]:second, [4]:meter\n\n# Font\ntitle_font_size: 14 \nlabel_font_size: 10\nticks_font_size: 8\n\n# Save\nsave_figures: true\nsave_extension_type: png # without \".\"\nsave_dataframe: true\noutput_directory: /path/to/output/directory\n\nuse_lerp: false\n```\n\n\u003c/details\u003e\n\nand run with\n\n```sh\n$ cd localization_evaluation_toolkit\n$ python3 scripts/pose_main.py config/evaluation_pose.yaml\n```\n\nFor example, you may output the following graphs with the sample data.\n\n```sh\n$ cd localization_evaluation_toolkit\n$ python3 scripts/pose_main.py sample_data/config/sasashima_evaluation.yaml\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e2D Trajectory\u003c/summary\u003e\n\u003cimg src=\"sample_data/output_sample/2d_trajectory.png\"\u003e\nWhen you zoom in on the graph, you can see the correspondence.\n\u003cimg src=\"sample_data/output_sample/2d_trajectory_zoom.png\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e3D Trajectory\u003c/summary\u003e\n\u003cimg src=\"sample_data/output_sample/3d_trajectory.png\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eX Y Z\u003c/summary\u003e\n\u003cimg src=\"sample_data/output_sample/xyz.png\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eX Y Z error\u003c/summary\u003e\n\u003cimg src=\"sample_data/output_sample/xyz_error.png\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e2D Error\u003c/summary\u003e\n\u003cimg src=\"sample_data/output_sample/2d_error.png\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e3D Error\u003c/summary\u003e\n\u003cimg src=\"sample_data/output_sample/3d_error.png\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eLongitudinal Error\u003c/summary\u003e\n\u003cimg src=\"sample_data/output_sample/longitudinal_error.png\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eLateral Error\u003c/summary\u003e\n\u003cimg src=\"sample_data/output_sample/lateral_error.png\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eRoll Pitch Yaw\u003c/summary\u003e\nYou can choose the unit of the vertical axis between radian and degree.\n\u003cimg src=\"sample_data/output_sample/rpy.png\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eRoll Pitch Yaw Error\u003c/summary\u003e\nYou can choose the unit of the vertical axis between radian and degree.\n\u003cimg src=\"sample_data/output_sample/rpy_error.png\"\u003e\n\u003c/details\u003e\n\n### 2.2 Twist evaluation\n\nA standard configuration file for twist evaluation consists of the following parts.\n\n- Reference\n- Twist\n- Optional settings\n\nIt is required that the reference block should have the name of ***Reference*** and the names of result blocks should begin with ***Twist***.\nWhen specifying the paths, it is recommended to use the absolute path.\nThere is a sample YAML file at `config/evaluation_twist.yaml`.\nTo create your own configuration file, just use the following templates,\n\n\u003cdetails\u003e\n\u003csummary\u003eTemplate of Reference (click to expand)\u003c/summary\u003e\n\n```yaml\nReference:\n  ## Auxiliary info\n  label: reference\n  path: /path/to/reference/csv\n\n  ## Time\n  separate_time_stamp: false # [true]:Set secs_stamp_column and nsecs_stamp_column / [false]:Set stamp_column\n  #--------true--------#\n  secs_stamp_column: 1\n  nsecs_stamp_column: 2\n  #--------false-------#\n  stamp_column: 2\n\n  ## Position\n  x_column: 4\n  y_column: 5\n  z_column: 6\n\n  ## Rotation\n  use_quaternion: true # [true]:Set Quaternion / [false]:Set Euler\n  #--------true--------#\n  # Quaternion\n  ori_x_column: 7\n  ori_y_column: 8\n  ori_z_column: 9\n  ori_w_column: 10\n  #--------false-------#\n  # Euler\n  use_radian: true # [true]:radian / [false]:degree\n  roll_column: 7\n  pitch_column: 8\n  yaw_column: 9\n\n  ## Enu velocity\n  use_enu_vel: false # [true]: use enu velocity / [false]: ignored\n  vel_x_column: 0\n  vel_y_column: 0\n  vel_z_column: 0\n\n  ## Angular\n  use_angular: false # [true]: use angular / [false]: ignored\n  angular_x_column: 0\n  angular_y_column: 0\n  angular_z_column: 0\n\n  # GNSS quality\n  use_gnss_qual: false # [true]: use GNSS quality / [false]: ignored\n  gnss_qual: 10\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eTemplate of Twist (click to expand)\u003c/summary\u003e\n\n```yaml\nTwist:\n  ## Auxiliary info\n  label: twist\n  path: /path/to/twist/csv\n\n  ## Time\n  separate_time_stamp: false # [true]:Set secs_stamp_column and nsecs_stamp_column / [false]:Set stamp_column\n  #--------true--------#\n  secs_stamp_column: 1\n  nsecs_stamp_column: 2\n  #--------false-------#\n  stamp_column: 2\n\n  ## Linear\n  linear_x_column: 4\n  linear_y_column: 5\n  linear_z_column: 6\n\n  ## Angular\n  angular_x_column: 7\n  angular_y_column: 8\n  angular_z_column: 9\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eTemplate of optional settings (click to expand)\u003c/summary\u003e\n\n```yaml\n# Trajectory graph numbering\nprogress_info: 0 # [0]:off, [1]:number, [2]:time, [3]:ros time, [4]:distance\ninterval: 0      # progress_info is [2]:second, [3]:second, [4]:meter\n\n# Misc\nsync_time_threshold: 0.01  # Time threshold for judgment for time synchronization [s]\nleap_time: 0.0             # Offset correction for time synchronizatio [s]\nbased_heading_angle: false # [true]:The heading angle is based on North / [false]:The heading angle is based on East (ros data)\ndistance_length: 100       # Distance to calculate relative trajectory [m]\ndistance_step: 50          # Calculate relative trajectories step [m]\neval_step_max: 3.0   # Maximum value of error to be evaluated default [m]\n\n# Font\ntitle_font_size: 14\nlabel_font_size: 10\nticks_font_size: 8\n\n# Save\nsave_figures: true\nsave_extension_type: png # without \".\"\nsave_dataframe: true\noutput_directory: /path/to/output/directory\n```\n\n\u003c/details\u003e\n\nand run with\n\n```sh\n$ cd localization_evaluation_toolkit\n$ python3 scripts/twist_main.py config/evaluation_twist.yaml\n```\n\n## 3. Sub-evaluation\n\n### 3.1 NDT performance\n\nEvaluation of TP, NVTL, execution time, iteration and error ellipse with ros2 bag files.\n\nThe input rosbag file should contain the following (at least one) topic.\n```\n/localization/pose_estimator/pose_with_covariance\n/localization/pose_estimator/nearest_voxel_transformation_likelihood\n/localization/pose_estimator/transform_probability\n/localization/pose_estimator/exe_time_ms\n/localization/pose_estimator/iteration_num\n```\n\n```sh\n$ cd localization_evaluation_toolkit/sub_scripts\n$ source ~/xxxxxx/install/setup.bash\n$ python3 sub_ndt_evaluation.py [bag_path] [output_folder_path]\n```\n\n### 3.2 Eagleye log evaluation\n\nCurrently, refer to [the original doc page](https://github.com/MapIV/eagleye/tree/main-ros1/eagleye_util/trajectory_plot#eagleye_pp_single_evaluation) for details.\n\n### 3.3 Covariance file generation\n\nCreate a CSV file with covariance columns from rosbag file. In detail, refer to [qiita article](https://qiita.com/koki2022/items/148d56e0f8eee45a0a62) for details.\n\n```sh\n$ cd localization_evaluation_toolkit/sub_scripts\n$ python covariance_to_csv.py [input_bag_path] [output_folder_path]\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmapiv%2Flocalization_evaluation_toolkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmapiv%2Flocalization_evaluation_toolkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmapiv%2Flocalization_evaluation_toolkit/lists"}