{"id":22745936,"url":"https://github.com/tumftm/mixnet","last_synced_at":"2025-04-14T10:07:18.903Z","repository":{"id":56812718,"uuid":"517739866","full_name":"TUMFTM/MixNet","owner":"TUMFTM","description":"Structured Deep Neural Motion Prediction of Opposing Vehicles for an Autonomous Racecar","archived":false,"fork":false,"pushed_at":"2024-11-20T09:12:42.000Z","size":5053,"stargazers_count":21,"open_issues_count":1,"forks_count":7,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-14T10:07:09.852Z","etag":null,"topics":["prediction-algorithm","ros2"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TUMFTM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-07-25T16:25:54.000Z","updated_at":"2025-03-20T07:58:10.000Z","dependencies_parsed_at":"2024-01-24T13:50:50.401Z","dependency_job_id":null,"html_url":"https://github.com/TUMFTM/MixNet","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUMFTM%2FMixNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUMFTM%2FMixNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUMFTM%2FMixNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUMFTM%2FMixNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TUMFTM","download_url":"https://codeload.github.com/TUMFTM/MixNet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248860270,"owners_count":21173342,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["prediction-algorithm","ros2"],"created_at":"2024-12-11T02:09:24.211Z","updated_at":"2025-04-14T10:07:18.876Z","avatar_url":"https://github.com/TUMFTM.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.6953977.svg)](https://doi.org/10.5281/zenodo.6953977)\n[![Linux](https://img.shields.io/badge/os-linux-blue.svg)](https://www.linux.org/)\n[![Docker](https://badgen.net/badge/icon/docker?icon=docker\u0026label)](https://www.docker.com/)\n[![ROS2galactic](https://img.shields.io/badge/ros2-galactic-blue.svg)](https://docs.ros.org/en/galactic/index.html)\n[![Python 3.8](https://img.shields.io/badge/python-3.8-blue.svg)](https://www.python.org/downloads/release/python-380/)\n# MixNet\n\n### Structured Deep Neural Motion Prediction for Autonomous Racing\n\n\u003cimg src=\"assets/mix_net_model.png\" alt=\"mix_net_model\" width=\"8000\"/\u003e\n\nThis repository provides a deep neural network approach to prediction that incorporates semantic knowledge to guarantee the quality of the output prediction. In this way, high robustness can be achieved without compromising prediction performance. The code is embedded into a ROS2-Node, which is configured in [mix_net](mix_net). The algorithm was developed by the team TUM Autonomous Motorsport for the [Indy Autonomous Challenge 2021](https://www.indyautonomouschallenge.com/) and the [Autonomous Challenge at CES 2022](https://www.indyautonomouschallenge.com/stream).\n\n## Requirements\n\n- Linux Ubuntu \u003e= 18.04\n- Docker \u003e= 18.04\n- Python \u003e= 3.8\n\n## Installation\n\nClone repository:\n```\ngit clone https://github.com/TUMFTM/MixNet.git\n```\n\nInstall requirements:\n```\npip install -r requirements.txt\n```\n\nInstall tex extensions, necessary to plot in with desired font:\n```\nsudo apt-get install texlive-latex-extra texlive-fonts-recommended dvipng cm-super\n```\n\n\n## MixNet training and evaluation\nA MixNet training can be started with the following command:\n```\npython train/mix_net_trainer.py\n```\nAll parameters used for the training and evaluation are specified in the MixNet-config-files ([train config](train/configs/mix_net/trainer_params_train.json), [test config](train/configs/mix_net/trainer_params_test.json)). Per default `10` exemplary scenarios are plotted to show the prediction behavior. The number of plotted scenarios can be changed by the flag `--max_plots`. Add `--plt_vel` to evaluate the velocity profile additionally. To save the plots instead of showing them, use the flag `--save_figs`, which stores the plots to [train/figs](train/figs). To run an evaluation of a trained net, add the argument `--test` and run the script again. \n\n## Data and Evaluation\n* A sample of the raw data ([data/mix_net_sample.pkl](data/mix_net_sample.pkl)) and the processed data (torch.Dataloader, [data/mix_net_dataloaders/sample](data/mix_net_dataloaders/sample)) are included in the repository. Per default, the samples are used in the MixNet trainer.\n\n* The full dataset, either raw data or the processed in torch.Dataloader-format can be downloaded [here](https://doi.org/10.5281/zenodo.6954020). To use the raw data, extract them and change the key `path` in the section `data` in the  [mix_net-config-file](train/configs/mix_net/trainer_params_test.json). Note that every parent directory containing .pkl-files can be specified. To use the dataloaders, extract them, set `\"from_saved_dataloader\": true` and specify the `dataloader_path` in the section `data` in the [mix_net-config-file](train/configs/mix_net/trainer_params_test.json).\n\n* The evaluation data is also available to [download](https://doi.org/10.5281/zenodo.6954020). Copy the folder `evaluation_data` into this repository folder `data`. The details how to evaluate the data are given in the related [readme](data/evaluation_data/README.md).\n\n## ROS2-Node\nIt is recommended to run the ROS2-node of the module in a Docker container. To build the related image, execute:\n```\ndocker build --pull --tag \u003cimage_name\u003e:\u003ctag\u003e .\n# e.g. docker build --pull --tag mix_net:0.0.1 .\n```\nTo run the container and launch the ROS2-node, run:\n```\ndocker run \u003cimage_name\u003e:\u003ctag\u003e ros2 launch mix_net mix_net.launch.py\n# e.g. docker run mix_net:0.0.1 ros2 launch mix_net mix_net.launch.py\n```\nAdd additional parameters to the ros2 launch command if desired, see section `Parameter and Files` below. For further details about Docker and ROS2, we refer to the official documentations.\n\n\n## Parameter and Files\n### Directory: `mix_net`\nThe directory [mix_net](mix_net) contains the ROS2-node to apply the MixNet in a full software stack. \n\n| Files | Description |\n| ------------------ | ----------------------- |\nmix_net/mix_net/mix_net_node.py | ROS2 main file to apply the MixNet\nmix_net/launch/mix_net.launch.py | ROS2 launch file with parameter definition\n\nThe [launch description](mix_net/launch/mix_net.launch.py) contains the following parameters:\n\n| Parameter | Type | Default | Description\n| ------------- | ------------- | ------ | ----- |\nfrequency | dynamic_typing | 20.0 | Cycle frequency in Hz of the node (int, float)|\ntrack | string | IMS | Name of used race track, currently only IMS supported |\nuse_sim_time | boolean | False | Flag to use sim time instead of system time to replay ROS-bags\nuse_cuda | boolean | False | If true the GPU and cuda is used\n\n\n### Directory: `tools`\nThe directory [tools](tools) contains the script to visualize logged data of the applied ROS2-Node. To visualize logged data of the mix_net-node run:\n```\npython tools/visualize_logfiles.py\n```\n\n### Directory: `train`\nThe directory [train](train) contains the scripts to train the MixNet and the benchmark model (IndyNet).\n\n| File | Description |\n| ------------------ | ----------------------- |\nmix_net_trainer.py | Training script of the MixNet\nmix_net_dataset.py | Dataset class, used in mix_net_trainer.py\nindy_net_trainer.py | Training script of the benchmark model (IndyNet)\nindy_net_bayes.py | Bayes optimizer of the benchmark model\nindy_net_evaluation.py | Evaluation script of the benchmark model\n\n## Module configuration\nThe configuration of the prediction module is set in [main_params.ini](mix_net/mix_net/config/main_params.ini). There are several options to specify the prediction behavior when applying the ROS2-node. A parameter description is given in the [README](mix_net/mix_net/config/README.md).\n\n\n## Qualitative Example\nBelow is an exemplary visualization of the prediction on a scenario on the Indianapolis Motor Speedway. The input features of history, left and right boundary as well as predicted and ground truth trajectory are shown. MixNet combines the best out of two worlds: the superposition of base curves and the comprehensive, learned scenario understanding.\n\u003cimg src=\"assets/smoothness.svg\" alt=\"smoothness\" width=\"9900\"/\u003e\n\n## Inference time\nThe average computation times for predicting four vehicles on a single core of an Intel i7-4720HQ 2.6 GHz CPU for MixNet is 9 ms.\n\n## References\nP. Karle, F. Török, M. Geisslinger and M. Lienkamp, \"MixNet: Physics Constrained Deep Neural Motion Prediction for Autonomous Racing,\" in *IEEE Access*, vol. 11, pp. 85914-85926, 2023, doi: 10.1109/ACCESS.2023.3303841.\n\nBibTex:\n```\n@ARTICLE{Karle2023_2,\n  author={Karle, Phillip and Török, Ferenc and Geisslinger, Maximilian and Lienkamp, Markus},\n  journal={IEEE Access}, \n  title={MixNet: Physics Constrained Deep Neural Motion Prediction for Autonomous Racing}, \n  year={2023},\n  volume={11},\n  number={},\n  pages={85914-85926},\n  doi={10.1109/ACCESS.2023.3303841}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftumftm%2Fmixnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftumftm%2Fmixnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftumftm%2Fmixnet/lists"}