{"id":15415466,"url":"https://github.com/dlr-mi/utrack","last_synced_at":"2025-04-15T21:23:20.024Z","repository":{"id":256532566,"uuid":"848189436","full_name":"DLR-MI/UTrack","owner":"DLR-MI","description":"Multi-Object Tracking with Uncertain Detections [ECCV 2024 UnCV]","archived":false,"fork":false,"pushed_at":"2024-09-26T13:36:24.000Z","size":197731,"stargazers_count":32,"open_issues_count":1,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-06T02:06:18.663Z","etag":null,"topics":["multi-object-tracking","multi-pedestrian-tracking","pytorch","tracking-by-detection","uncertainty-estimation","uncertainty-propagation","uncertainty-quantification","yolov8"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DLR-MI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-27T09:52:56.000Z","updated_at":"2024-09-30T12:25:46.000Z","dependencies_parsed_at":"2024-10-26T08:34:12.108Z","dependency_job_id":"322aa70d-299b-4006-8633-3885e2050695","html_url":"https://github.com/DLR-MI/UTrack","commit_stats":{"total_commits":13,"total_committers":3,"mean_commits":4.333333333333333,"dds":"0.15384615384615385","last_synced_commit":"b14cd7f825462c3a0a4ab73459f6e173f5e2ea1b"},"previous_names":["dlr-mi/utrack"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DLR-MI%2FUTrack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DLR-MI%2FUTrack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DLR-MI%2FUTrack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DLR-MI%2FUTrack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DLR-MI","download_url":"https://codeload.github.com/DLR-MI/UTrack/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249155111,"owners_count":21221540,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["multi-object-tracking","multi-pedestrian-tracking","pytorch","tracking-by-detection","uncertainty-estimation","uncertainty-propagation","uncertainty-quantification","yolov8"],"created_at":"2024-10-01T17:08:29.235Z","updated_at":"2025-04-15T21:23:19.998Z","avatar_url":"https://github.com/DLR-MI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# UTrack\n\n[![arXiv](https://img.shields.io/badge/arXiv-2408.17098-\u003cCOLOR\u003e.svg)](https://arxiv.org/abs/2408.17098) [![License: MIT](https://img.shields.io/badge/License-AGPLv3-yellow.svg)](https://www.gnu.org/licenses/agpl-3.0) ![test](https://img.shields.io/static/v1?label=By\u0026message=Pytorch\u0026color=red)\n\n\u003e [**UTrack: Multi-Object Tracking with Uncertain Detections**](https://arxiv.org/abs/2408.17098)\n\u003e \n\u003e Edgardo Solano-Carrillo, Felix Sattler, Antje Alex, Alexander Klein, Bruno Pereira Costa, Angel Bueno Rodriguez, Jannis Stoppe\n\n\n## Abstract \n\nThe tracking-by-detection paradigm is the mainstream in\nmulti-object tracking, associating tracks to the predictions of an object\ndetector. Although exhibiting uncertainty through a confidence score,\nthese predictions do not capture the entire variability of the inference\nprocess. For safety and security critical applications like autonomous\ndriving, surveillance, etc., knowing this predictive uncertainty is essential though.\nTherefore, we introduce, for the first time, a fast way to\nobtain the empirical predictive distribution during object detection and\nincorporate that knowledge in multi-object tracking. Our mechanism can\neasily be integrated into state-of-the-art trackers, enabling them to fully\nexploit the uncertainty in the detections. Additionally, novel association\nmethods are introduced that leverage the proposed mechanism. We\ndemonstrate the effectiveness of our contribution on a variety of benchmarks,\nsuch as MOT17, MOT20, DanceTrack, and KITTI.\n\n\u003cimg src=\"assets/dancetrack_var.gif\" width=\"400\"/\u003e\n\u003cimg src=\"assets/var_gradient.png\"/\u003e\n\n\n## Highlights 🚀\n\n- YOLOv8 support using a fork of [Ultralytics](https://github.com/DLR-MI/ultralytics/tree/nms-var)\n- Non maximum suppresion (NMS) with box variances using [NMS-var](https://github.com/DLR-MI/nms_var)\n- Fast camera motion compensation (affine \u0026 homography) using [FastGMC](https://github.com/DLR-MI/fast_gmc)\n- Estimates of errors in IoU using [fuzzy cython-bbox](https://github.com/DLR-MI/fuzzy_cython_bbox)\n- Visualize metrics using [Dash](https://dash.plotly.com/)\n- Several integrated trackers\n  - [x] ByteTrack\n  - [x] BoT-SORT\n  - [x] SparseTrack\n  - [x] UTrack \n  - [ ] UCMCTrack (experimental)\n  - [ ] KCM-Track (experimental)\n\n## Installation\nFirst clone this repository:\n```shell\ngit clone https://github.com/DLR-MI/UTrack\ncd UTrack\n```\nand install suitable dependencies. This was tested on conda environment with Python 3.8.\n```shell\nmamba create -n track python=3.8\nmamba activate track\nmamba install pytorch torchvision\npip install cython\npip install -r requirements.txt\n```\n\n\n## Datasets\nDownload [MOT17](https://motchallenge.net/), [MOT20](https://motchallenge.net/), [CrowdHuman](https://www.crowdhuman.org/), [Cityperson](https://github.com/Zhongdao/Towards-Realtime-MOT/blob/master/DATASET_ZOO.md), [ETHZ](https://github.com/Zhongdao/Towards-Realtime-MOT/blob/master/DATASET_ZOO.md) and put them in a `/data` folder. Prepare the datasets for training MOT17 and MOT20 for ablation and testing:\n\n```shell\nbash tools/convert_datasets_to_coco.sh\nbash tools/mix_data_for_training.sh\n```\n\nIf you use another path for the data, make sure to change the `DATA_PATH` in the scripts invoked above. To clip the annotated bounding boxes to the image, run (e.g. for MOT17)\n\n```shell\npython tools/fix_yolo_annotations.py --folder /data/MOT17\n```\n\nFor [DanceTrack](https://dancetrack.github.io/) and [KITTI](https://www.cvlibs.net/datasets/kitti/eval_tracking.php), you can also find scripts in `./tools` to convert to the COCO format. No mixing is necessary here. \n\n\u003e **Note:** make sure that the folder structure of KITTI, after converting to COCO, mimics the one for MOTXX.\n\n## Training\n\nAs an example, training `YOLOv8-l` for the ablation experiments of MOT17 is done by running\n\n```shell\n$ python train.py --model yolov8l --exp ablation_17 --gpu_id 0\n```\nTake a look at `train.py` for the available `mot_choices` for the `--exp` argument.\n\nThe model weights used for the experiments in the paper can be downloaded below.\n\n| Dataset | Weights|\n|-----| ------|\n| Ablation 17 |  [[download]](https://zenodo.org/records/13604403/files/ablation_17_best.pt?download=1) |\n| Ablation 20 |  [[download]](https://zenodo.org/records/13604403/files/ablation_20_best.pt?download=1) |\n| Mix 17 |  [[download]](https://zenodo.org/records/13604403/files/mix_17_best.pt?download=1) |\n| Mix 20 |  [[download]](https://zenodo.org/records/13604403/files/mix_20_best.pt?download=1) |\n| DanceTrack |  [[download]](https://zenodo.org/records/13604403/files/dancetrack_best.pt?download=1) |\n| KITTI |  [[download]](https://zenodo.org/records/13604403/files/kitti_best.pt?download=1) |\n\nAfter downloading, rename to `best.pt` and place it within the corresponding folder in `./yolov8l-mix/EXP/weights`, where `EXP` is the experiment name referred to during training (i.e. `mix_17`, `ablation_17`, etc.) Note that KITTI was only trained for the pedestrian class.\n\n## Evaluation\nFor tracking, you just need to modify `./tracker/config/track_EXP.yaml` if considering different tracker parameters, where `EXP` is the experiment name. \n\n```shell\n$ python track.py --project yolov8l-mix --exp ablation_17 --data_root /data/MOT17 --association uk_botsort\n```\n\nThis produces an entry in the folder `./track_results` with relevant metrics. The available trackers can be explored\nby referring to the `./tracker/associations/collections.py` module.\n\nYou can do hyperparameter search of multiple trackers in parallel, or run multiple seeds for a single tracker. The former is done by executing `./hp_search.py` on a `.yaml` template in `./tracker/eval/hp_search`. The latter, by using\nthe `--association` argument.\n\n\n## Results on DanceTrack test set\n\nJust by adding (on top of BoT-SORT) the right observation noise in the Kalman filter:\n\n\n|  Method  | HOTA | DetA | AssA | MOTA | IDF1 |\n|------------|-------|-------|------|------|-------|\n| BoT-SORT | 53.8 | 77.8 | 37.3 | 89.7 | 56.1 |\n| UTrack | 55.8 | 79.4 | 39.3 | 89.7 | 56.4 |\n\nThe evaluation runs at **27 FPS** on a NVIDIA A100 GPU, compared with the 32 FPS of BoT-SORT in the same machine.\n\n### Demo\n\nA demo can be run on a sample video downloaded from [Youtube](https://www.youtube.com/watch?v=qv6gl4h0dvg) by executing\n\n```shell\npython ./tools/track_demo.py --exp dancetrack --association uk_botsort --video_path /path/to/cropped_video --output_path /path/to/output/video\n```\n\n\u003cimg src=\"assets/dance_sample_track.gif\" width=\"400\"/\u003e\n\n\n## Citation\nIf you find this work useful, it would be cool to know it by giving us a star 🌟. Also, consider citing it as\n\n\n```bibtex\n@misc{UTrack,\n      title={UTrack: Multi-Object Tracking with Uncertain Detections}, \n      author={Edgardo Solano-Carrillo and Felix Sattler and Antje Alex and Alexander Klein and Bruno Pereira Costa and Angel Bueno Rodriguez and Jannis Stoppe},\n      year={2024},\n      eprint={2408.17098},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV},\n      url={https://arxiv.org/abs/2408.17098}, \n}\n```\n\n## Acknowledgements\nA portion of the code is borrowed from [ByteTrack](https://github.com/ifzhang/ByteTrack), [BoT-SORT](https://github.com/NirAharon/BOT-SORT), [SparseTrack](https://github.com/hustvl/SparseTrack/tree/main), [UCMCTrack](https://github.com/corfyi/UCMCTrack), and [TrackEval](https://github.com/JonathonLuiten/TrackEval). \nMany thanks for their contributions.\n\nAlso thanks to [Ultralytics](https://github.com/ultralytics/ultralytics) for making object detection more user friendly. \n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdlr-mi%2Futrack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdlr-mi%2Futrack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdlr-mi%2Futrack/lists"}