https://github.com/tumftm/mixnet
Structured Deep Neural Motion Prediction of Opposing Vehicles for an Autonomous Racecar
https://github.com/tumftm/mixnet
prediction-algorithm ros2
Last synced: 8 months ago
JSON representation
Structured Deep Neural Motion Prediction of Opposing Vehicles for an Autonomous Racecar
- Host: GitHub
- URL: https://github.com/tumftm/mixnet
- Owner: TUMFTM
- License: gpl-3.0
- Created: 2022-07-25T16:25:54.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2024-11-20T09:12:42.000Z (about 1 year ago)
- Last Synced: 2025-04-14T10:07:09.852Z (8 months ago)
- Topics: prediction-algorithm, ros2
- Language: Python
- Homepage:
- Size: 4.82 MB
- Stars: 21
- Watchers: 4
- Forks: 7
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
[](https://doi.org/10.5281/zenodo.6953977)
[](https://www.linux.org/)
[](https://www.docker.com/)
[](https://docs.ros.org/en/galactic/index.html)
[](https://www.python.org/downloads/release/python-380/)
# MixNet
### Structured Deep Neural Motion Prediction for Autonomous Racing
This repository provides a deep neural network approach to prediction that incorporates semantic knowledge to guarantee the quality of the output prediction. In this way, high robustness can be achieved without compromising prediction performance. The code is embedded into a ROS2-Node, which is configured in [mix_net](mix_net). The algorithm was developed by the team TUM Autonomous Motorsport for the [Indy Autonomous Challenge 2021](https://www.indyautonomouschallenge.com/) and the [Autonomous Challenge at CES 2022](https://www.indyautonomouschallenge.com/stream).
## Requirements
- Linux Ubuntu >= 18.04
- Docker >= 18.04
- Python >= 3.8
## Installation
Clone repository:
```
git clone https://github.com/TUMFTM/MixNet.git
```
Install requirements:
```
pip install -r requirements.txt
```
Install tex extensions, necessary to plot in with desired font:
```
sudo apt-get install texlive-latex-extra texlive-fonts-recommended dvipng cm-super
```
## MixNet training and evaluation
A MixNet training can be started with the following command:
```
python train/mix_net_trainer.py
```
All parameters used for the training and evaluation are specified in the MixNet-config-files ([train config](train/configs/mix_net/trainer_params_train.json), [test config](train/configs/mix_net/trainer_params_test.json)). Per default `10` exemplary scenarios are plotted to show the prediction behavior. The number of plotted scenarios can be changed by the flag `--max_plots`. Add `--plt_vel` to evaluate the velocity profile additionally. To save the plots instead of showing them, use the flag `--save_figs`, which stores the plots to [train/figs](train/figs). To run an evaluation of a trained net, add the argument `--test` and run the script again.
## Data and Evaluation
* A sample of the raw data ([data/mix_net_sample.pkl](data/mix_net_sample.pkl)) and the processed data (torch.Dataloader, [data/mix_net_dataloaders/sample](data/mix_net_dataloaders/sample)) are included in the repository. Per default, the samples are used in the MixNet trainer.
* The full dataset, either raw data or the processed in torch.Dataloader-format can be downloaded [here](https://doi.org/10.5281/zenodo.6954020). To use the raw data, extract them and change the key `path` in the section `data` in the [mix_net-config-file](train/configs/mix_net/trainer_params_test.json). Note that every parent directory containing .pkl-files can be specified. To use the dataloaders, extract them, set `"from_saved_dataloader": true` and specify the `dataloader_path` in the section `data` in the [mix_net-config-file](train/configs/mix_net/trainer_params_test.json).
* The evaluation data is also available to [download](https://doi.org/10.5281/zenodo.6954020). Copy the folder `evaluation_data` into this repository folder `data`. The details how to evaluate the data are given in the related [readme](data/evaluation_data/README.md).
## ROS2-Node
It is recommended to run the ROS2-node of the module in a Docker container. To build the related image, execute:
```
docker build --pull --tag : .
# e.g. docker build --pull --tag mix_net:0.0.1 .
```
To run the container and launch the ROS2-node, run:
```
docker run : ros2 launch mix_net mix_net.launch.py
# e.g. docker run mix_net:0.0.1 ros2 launch mix_net mix_net.launch.py
```
Add additional parameters to the ros2 launch command if desired, see section `Parameter and Files` below. For further details about Docker and ROS2, we refer to the official documentations.
## Parameter and Files
### Directory: `mix_net`
The directory [mix_net](mix_net) contains the ROS2-node to apply the MixNet in a full software stack.
| Files | Description |
| ------------------ | ----------------------- |
mix_net/mix_net/mix_net_node.py | ROS2 main file to apply the MixNet
mix_net/launch/mix_net.launch.py | ROS2 launch file with parameter definition
The [launch description](mix_net/launch/mix_net.launch.py) contains the following parameters:
| Parameter | Type | Default | Description
| ------------- | ------------- | ------ | ----- |
frequency | dynamic_typing | 20.0 | Cycle frequency in Hz of the node (int, float)|
track | string | IMS | Name of used race track, currently only IMS supported |
use_sim_time | boolean | False | Flag to use sim time instead of system time to replay ROS-bags
use_cuda | boolean | False | If true the GPU and cuda is used
### Directory: `tools`
The directory [tools](tools) contains the script to visualize logged data of the applied ROS2-Node. To visualize logged data of the mix_net-node run:
```
python tools/visualize_logfiles.py
```
### Directory: `train`
The directory [train](train) contains the scripts to train the MixNet and the benchmark model (IndyNet).
| File | Description |
| ------------------ | ----------------------- |
mix_net_trainer.py | Training script of the MixNet
mix_net_dataset.py | Dataset class, used in mix_net_trainer.py
indy_net_trainer.py | Training script of the benchmark model (IndyNet)
indy_net_bayes.py | Bayes optimizer of the benchmark model
indy_net_evaluation.py | Evaluation script of the benchmark model
## Module configuration
The configuration of the prediction module is set in [main_params.ini](mix_net/mix_net/config/main_params.ini). There are several options to specify the prediction behavior when applying the ROS2-node. A parameter description is given in the [README](mix_net/mix_net/config/README.md).
## Qualitative Example
Below is an exemplary visualization of the prediction on a scenario on the Indianapolis Motor Speedway. The input features of history, left and right boundary as well as predicted and ground truth trajectory are shown. MixNet combines the best out of two worlds: the superposition of base curves and the comprehensive, learned scenario understanding.
## Inference time
The average computation times for predicting four vehicles on a single core of an Intel i7-4720HQ 2.6 GHz CPU for MixNet is 9 ms.
## References
P. Karle, F. Török, M. Geisslinger and M. Lienkamp, "MixNet: Physics Constrained Deep Neural Motion Prediction for Autonomous Racing," in *IEEE Access*, vol. 11, pp. 85914-85926, 2023, doi: 10.1109/ACCESS.2023.3303841.
BibTex:
```
@ARTICLE{Karle2023_2,
author={Karle, Phillip and Török, Ferenc and Geisslinger, Maximilian and Lienkamp, Markus},
journal={IEEE Access},
title={MixNet: Physics Constrained Deep Neural Motion Prediction for Autonomous Racing},
year={2023},
volume={11},
number={},
pages={85914-85926},
doi={10.1109/ACCESS.2023.3303841}
}
```