{"id":22746001,"url":"https://github.com/tumftm/fusiontracking","last_synced_at":"2025-04-14T10:07:49.188Z","repository":{"id":65571760,"uuid":"553471360","full_name":"TUMFTM/FusionTracking","owner":"TUMFTM","description":"Multi-Modal Sensor Fusion and Object Tracking for Autonomous Racing","archived":false,"fork":false,"pushed_at":"2023-08-16T15:08:00.000Z","size":576,"stargazers_count":83,"open_issues_count":1,"forks_count":13,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-14T10:07:44.614Z","etag":null,"topics":["fusion","kalman-filter","python","ros2","ros2-node"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TUMFTM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-10-18T09:11:04.000Z","updated_at":"2025-04-13T20:58:40.000Z","dependencies_parsed_at":"2023-02-01T06:45:42.675Z","dependency_job_id":null,"html_url":"https://github.com/TUMFTM/FusionTracking","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUMFTM%2FFusionTracking","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUMFTM%2FFusionTracking/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUMFTM%2FFusionTracking/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUMFTM%2FFusionTracking/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TUMFTM","download_url":"https://codeload.github.com/TUMFTM/FusionTracking/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248860268,"owners_count":21173342,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fusion","kalman-filter","python","ros2","ros2-node"],"created_at":"2024-12-11T02:09:51.379Z","updated_at":"2025-04-14T10:07:49.157Z","avatar_url":"https://github.com/TUMFTM.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7220719.svg)](https://doi.org/10.5281/zenodo.7220719)\n[![Linux](https://img.shields.io/badge/os-linux-blue.svg)](https://www.linux.org/)\n[![Docker](https://badgen.net/badge/icon/docker?icon=docker\u0026label)](https://www.docker.com/)\n[![ROS2galactic](https://img.shields.io/badge/ros2-galactic-blue.svg)](https://docs.ros.org/en/galactic/index.html)\n[![Python 3.8](https://img.shields.io/badge/python-3.8-blue.svg)](https://www.python.org/downloads/release/python-380/)\n\n# Multi-Modal Sensor Fusion and Object Tracking\nThe following figure outlines the high level structure of the algorithm, which covers the tasks of multi-modal sensor fusion and object tracking. The algorithm is developed for the [Indy Autonomous Challenge 2021](https://www.indyautonomouschallenge.com/) and the [Autonomous Challenge at CES 2022](https://www.indyautonomouschallenge.com/stream) and is part of the software of TUM Autonomous Motorsport.\n\n\u003cimg src=\"assets/tracking_structure.svg\" alt=\"Overview of object fusion and tracking\" width=\"9900\"/\u003e\n\nThe sensor fusion handles multiple object lists that originate from different perception pipelines. The perception pipelines work independently from each other and output individual object lists. This algorithm combines the given information to output a unified object list. This late fusion approach allows us to incorporate a variable number of perception pipelines without any dependencies. \n\nThe object tracking addresses the estimation of the detected objects' dynamic states, which is realized by an Extended Kalman Filter (EKF) based on a constant turn-rate and velocity (CTRV)-model.\n\n### Requirements\n- OS: `Ubuntu 22.04 LTS`\n- Docker: `20.10.17`\n- Docker Compose: `v2.6.1`\n- Python: `3.8`\n- ROS2: `galactic`\n\n## Installation\n\nClone repository:\n```\ngit clone https://github.com/TUMFTM/FusionTacking.git\n```\n\nSetup virtual environment and install requirements:\n```\npython3.8 -m venv venv\nsource venv/bin/activate\npip install -r requirements.txt\n```\n\nInstall tex extensions, necessary to plot with desired font:\n```\nsudo apt-get install texlive-latex-extra texlive-fonts-recommended dvipng cm-super\n```\n\n\n## Data and Evaluation\nThe evaluation is entirely conducted with real-world data of team TUM Autonomous Motorsport from the [AC@CES 2022](https://www.indyautonomouschallenge.com/stream). The recorded raw data of all tracking inputs stored in rosbags is available open-source ([Link](https://doi.org/10.5281/zenodo.7220695), uncompressed size: 16.4 GB). The data processing and evaluation procedure is described in the [README](evaluation/README.md). Follow the described steps to reproduce the evaluation.\n\n\n## Docker Image\nIt is recommended to run the ROS2 node of the module in a Docker container. To build the related image, execute:\n```\ndocker build --pull --tag \u003cimage_name\u003e:\u003ctag\u003e .\n# e.g. docker build --pull --tag tracking:0.0.1 .\n```\nTo run the container and launch the ROS2 node, run:\n```\ndocker run \u003cimage_name\u003e:\u003ctag\u003e ros2 launch tracking tracking.launch.py\n# e.g. docker run tracking:0.0.1 ros2 launch tracking tracking.launch.py\n```\nIt is recommended to mount a volume to save the logs during the node runs (see [replay.yml](evaluation/bag_play/replay.yml) for an example).\nAdd additional parameters to the ros2 launch command if desired, see section `Parameter and Files` below. For further details about Docker and ROS2, we refer to the official documentations.\n\n\n\n## Parameter and Files\n### Directory: `tracking`\nThe directory `tracking` contains the source code (subfolder: `tracking`) and ROS2 launch configuration (subfolder: `launch`) of the module. \n\n| Files | Description |\n| ------------------ | ----------------------- |\n`tracking/tracking_node.py` | ROS2 main file to apply the MixNet\n`launch/tracking.launch.py` | ROS2 launch file with parameter definition\n\nThe [launch description](tracking/launch/tracking.launch.py) contains the following parameters:\n\n| Parameter | Type | Default | Description\n| ------------- | ------------- | ----- | ----- |\nfrequency | float, int | 50.0 Hz | Cycle frequency of the ROS2 node |\nmax_delay_ego_s | float | 0.15 s | Threshold for ego state message delay |\nchecks_enabled | boolean | False | If true failed safety checks trigger emergency state of the module |\ntrack | string | LVMS | Name of used race track map |\nuse_sim_time | boolean | False | Flag to use sim time instead of system time\nego_raceline | string | default | String of used ego raceline of motion planner (default, inner, outer, center)\nsend_prediction | boolean | True | If true a prediction is published\n\nAdd them at the end of the `docker run`-command. Example with modified frequency and enabled safety checks:\n```\ndocker run tracking:0.0.1 ros2 launch tracking tracking.launch.py frequency:=100.0 checks_enabled:=True\n```\n\n### Directory: `tools`\nThe directory `tools` contains the script to visualize logged data of the applied ROS2 Node. To visualize logged data of the tracking-node run:\n```\npython tools/visualize_logfiles.py\n```\nLogs must be stored in `tracking/tracking/logs` to be considered. Enter the number of the desired log or hit `enter` to run the latest.\nAdd additional arguments if desired. Without any argument the overall tracking process will be shown (always recommended at first).\n\nAdditional Arguments:\n- `--n_obs`: Specifies the number of objects to show filter values / states of (default: 5)\n- `--filter`: Visualizes filter values of the \u003cn_obs\u003e-most seen objects (default: False)\n- `--states`: Visualizes the dynamic states of the \u003cn_obs\u003e-most seen objects (default: False)\n- `--mis_num_obj`: Visualizes the matching performance (default: False)\n\n## Module configuration\nBesides the ROS2 parameter, which are changeable during runtime, there is an additional configuration file for all static parameters (i.e. not changeable during runtime). This config is stored in [main_config_tracking.ini](tracking/tracking/config/main_config_tracking.ini). A parameter description is given in the related [README](tracking/tracking/config/README.md).\n\n\n## Qualitative Example\nBelow is an exemplary visualization of the fusion and tracking algorithm on the Las Vegas Motor Speedway. The input are the object lists of the LiDAR and the RADAR.\nThe raw perception input is filtered for objects outside the driveable area and it is checked if there are multiple detections of the same object. Afterwards a distance-based matching associates the multi-modal object lists to the currently tracked objects. A kinematic tracking model based on an Extended Kalman Filter (EKF) is applied to optimize the state estimation. Additionally, the perception delay is compensated by backward-forward integration of the kinematic model to output an updated object list.\nThe validation under real-world condition in autonomous racing with a stable object fusion and tracking at a maximum speed of 270 km/h proves the robustness of the proposed approach.\n\n\u003cimg src=\"assets/delay_compensation_45790.svg\" alt=\"Example of Fusion and Tracking module.\" width=\"9900\"/\u003e\n\n## Inference time\nThe average computation time of the module during the evaluation on the given data on a single core of an Intel i7-9850H 2.60 GHz CPU is 2.75 ms with a 90%-quantile of 5.26 ms.\n\n## References\nP. Karle, F. Fent, S. Huch, F. Sauerbeck and M. Lienkamp, \"Multi-Modal Sensor Fusion and Object Tracking for Autonomous Racing,\" in IEEE Transactions on Intelligent Vehicles, doi: 10.1109/TIV.2023.3271624.\n\nBibTex:\n```\n@ARTICLE{Karle2023,\n  author={Karle, Phillip and Fent, Felix and Huch, Sebastian and Sauerbeck, Florian and Lienkamp, Markus},\n  journal={IEEE Transactions on Intelligent Vehicles}, \n  title={Multi-Modal Sensor Fusion and Object Tracking for Autonomous Racing}, \n  year={2023},\n  pages={1-13},\n  doi={10.1109/TIV.2023.3271624}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftumftm%2Ffusiontracking","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftumftm%2Ffusiontracking","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftumftm%2Ffusiontracking/lists"}