{"id":13704800,"url":"https://github.com/yihongXU/deepMOT","last_synced_at":"2025-05-05T12:32:32.388Z","repository":{"id":47675919,"uuid":"192505017","full_name":"yihongXU/deepMOT","owner":"yihongXU","description":"Official Implementation of How To Train Your Deep Multi-Object Tracker (CVPR2020)","archived":false,"fork":false,"pushed_at":"2023-03-20T20:06:09.000Z","size":29657,"stargazers_count":498,"open_issues_count":4,"forks_count":85,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-01-19T23:39:25.288Z","etag":null,"topics":["deep-learning","multi-object-tracking","python","pytorch"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yihongXU.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-06-18T09:06:09.000Z","updated_at":"2024-12-21T13:12:22.000Z","dependencies_parsed_at":"2022-08-21T16:40:32.534Z","dependency_job_id":"cd7e37ff-bdc6-49d8-9310-5e0bdb2df4fc","html_url":"https://github.com/yihongXU/deepMOT","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yihongXU%2FdeepMOT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yihongXU%2FdeepMOT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yihongXU%2FdeepMOT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yihongXU%2FdeepMOT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yihongXU","download_url":"https://codeload.github.com/yihongXU/deepMOT/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252497653,"owners_count":21757655,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","multi-object-tracking","python","pytorch"],"created_at":"2024-08-02T22:00:17.331Z","updated_at":"2025-05-05T12:32:28.161Z","avatar_url":"https://github.com/yihongXU.png","language":null,"funding_links":[],"categories":["算法论文"],"sub_categories":["**2019**"],"readme":"## CVPR2020: How To Train Your Deep Multi-Object Tracker \u003cbr /\u003e\n[![License: LGPL v3](https://img.shields.io/badge/License-LGPL%20v3-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0)\n\n**News: We release the code for training and testing DeepMOT-Tracktor and the code for training DHN. Please visit: https://gitlab.inria.fr/robotlearn/deepmot**\n\n**How To Train Your Deep Multi-Object Tracker** \u003cbr /\u003e\n[Yihong Xu](https://team.inria.fr/perception/team-members/yihong-xu/), [Aljosa Osep](https://dvl.in.tum.de/team/osep/), [Yutong Ban](https://team.inria.fr/perception/team-members/yutong-ban/), [Radu Horaud](https://team.inria.fr/perception/team-members/radu-patrice-horaud/), [Laura Leal-Taixé](https://dvl.in.tum.de/team/lealtaixe/), [Xavier Alameda-Pineda](https://team.inria.fr/perception/team-members/xavier-alameda-pineda/) \u003cbr /\u003e\n**[[Paper](https://arxiv.org/abs/1906.06618)]** \u003cbr /\u003e\n\n### Bibtex\nIf you find this code useful, please star the project and consider citing:\n\n```\n@inproceedings{xu2020train,\n  title={How To Train Your Deep Multi-Object Tracker},\n  author={Xu, Yihong and Osep, Aljosa and Ban, Yutong and Horaud, Radu and Leal-Taix{\\'e}, Laura and Alameda-Pineda, Xavier},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={6787--6796},\n  year={2020}\n}\n```\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://gitlab.inria.fr/yixu/deepmot/-/raw/master/teaser.pdf\" width=\"800px\" /\u003e\n\u003c/div\u003e\n\n## Environment setup \u003ca name=\"environment-setup\"\u003e\nThis code has been tested on Ubuntu 16.04, Python 3.6, Pytorch=0.4.1, CUDA 9.2, GTX 1080Ti, Titan X, and RTX Titan GPUs.\n\n**Warning: the results can be slightly different due to Pytorch version and CUDA version.**\n\n- Clone the repository \n```\ngit clone git@gitlab.inria.fr:yixu/deepmot.git \u0026\u0026 cd deepmot\n```\n**Option 1:**\n- Follow the installation instructions in [**Tracktor**](https://github.com/phil-bergmann/tracking_wo_bnw/tree/iccv_19).\u003cbr/\u003e\n\n**Option 2 (recommended):**\n\nwe provide a Singularity image with all packages pre-installed (similar to Docker) for training and testing.\n- Open a terminal\n- Install Singularity \u003e 3.0 package: \u003cbr /\u003e\n[https://sylabs.io/guides/3.3/user-guide/installation.html#install-on-linux](https://sylabs.io/guides/3.3/user-guide/installation.html#install-on-linux)\n- Download the Singularity image: \u003cbr /\u003e\n[tracker.sif (google drive)](https://drive.google.com/file/d/1sR-tTtprbkQ1NAIpY2oiQjSrbuxoYTCc/view?usp=sharing)  or \u003cbr /\u003e\n[tracker.sif (tencent cloud)](https://share.weiyun.com/5RK76iq) \u003cbr /\u003e\n- Open a new terminal\n- Launch a Singularity image\n```shell\nsingularity shell --nv --bind yourLocalPath:yourPathInsideImage tracker.sif\n```\n**- -bind: to link a singularity path with a local path. By doing this, you can find data from local PC inside Singularity image;** \u003cbr /\u003e\n**- -nv: use the local Nvidia driver.**\n\n## Testing \u003ca name=\"testing-models\"\u003e\n- [Setup](#environment-setup) your environment\n- Go to the test_tracktor folder\n- Download MOT data\nDataset can be downloaded here: [MOT17Det](https://motchallenge.net/data/MOT17Det.zip), [MOT16Labels](https://motchallenge.net/data/MOT16Labels.zip), [MOT16-det-dpm-raw](https://motchallenge.net/data/MOT16-det-dpm-raw.zip) and [MOT17Labels](https://motchallenge.net/data/MOT17Labels.zip) .\n    2. Unzip all the data by executing:\n    ```\n    unzip -d MOT17Det MOT17Det.zip\n    unzip -d MOT16Labels MOT16Labels.zip\n    unzip -d 2DMOT2015 2DMOT2015.zip\n    unzip -d MOT16-det-dpm-raw MOT16-det-dpm-raw.zip\n    unzip -d MOT17Labels MOT17Labels.zip\n    ```\n- Enter the data path to *data_pth* in the *test_tracktor/experiments/cfgs/tracktor_pub_reid.yaml* and *test_tracktor/experiments/cfgs/tracktor_private.yaml* \u003cbr /\u003e\n\n- Download pretrained models\nall the pretrained models can be downloaded here: \u003cbr /\u003e\n[deepMOT-Tracktor.pth (google drive)](https://drive.google.com/file/d/181JzMrK5YyGecEZkKj-MPuAx2QLqSryO/view?usp=sharing) or \u003cbr /\u003e\n[deepMOT-Tracktor.pth (tencent cloud)](https://share.weiyun.com/5ZXIUL6)\n\n- Enter the model path to parameter *obj_detect_weights* in the *test_tracktor/experiments/cfgs/tracktor_pub_reid.yaml* and *test_tracktor/experiments/cfgs/tracktor_private.yaml* \u003cbr /\u003e\n- Set the dataset name in the test_tracktor/experiments/cfgs/tracktor_pub_reid.yaml and test_tracktor/experiments/cfgs/tracktor_private.yaml: \u003cbr /\u003e\n   For MOT17 (by default):  \n```\ndataset: mot17_train_17\n```\n\n   For MOT16 (images as the same as MOT17):  \n```\ndataset: mot17_all_DPM_RAW16\n```\n\n- run tracking code\n```\npython test_tracktor/experiments/scripts/tst_tracktor_private.pytst_tracktor_pub_reid.py (public detections) or test_tracktor/experiments/scripts/tst_tracktor_private.py (private detections)\n```\n\nThe results are saved by default under *test_tracktor/output/log/*, you can modify it by changing *output_dir* in the *test_tracktor/experiments/cfgs/tracktor_pub_reid.yaml* and *test_tracktor/experiments/cfgs/tracktor_private.yaml*.\n\n- Visualization: \u003cbr/\u003e\nYou can set write_images: True in the test_tracktor/experiments/cfgs/tracktor_pub_reid.yaml and test_tracktor/experiments/cfgs/tracktor_private.yaml to plot and save images.\n By default, they will be saved inside *test_tracktor/output/log/* if *write_images: True*.\n\n\n## Training \u003ca name=\"training-models\"\u003e\n\n- [Setup](#environment-setup) your environment\n- Go to the train_tracktor folder\n- Download MOT Dataset can be downloaded here: [MOT17Det](https://motchallenge.net/data/MOT17Det.zip), [MOT16Labels](https://motchallenge.net/data/MOT16Labels.zip), [MOT16-det-dpm-raw](https://motchallenge.net/data/MOT16-det-dpm-raw.zip) and [MOT17Labels](https://motchallenge.net/data/MOT17Labels.zip).\n- Unzip all the data by executing:\n    ```\n    unzip -d MOT17Det MOT17Det.zip\n    unzip -d MOT16Labels MOT16Labels.zip\n    unzip -d 2DMOT2015 2DMOT2015.zip\n    unzip -d MOT16-det-dpm-raw MOT16-det-dpm-raw.zip\n    unzip -d MOT17Labels MOT17Labels.zip\n    ```\n- Enter the data path to *data_pth* in the *train_tracktor/experiments/cfgs/tracktor_full.yaml*\n\n- Download the output folder containing the configurations and the model to be fine-tuned and DHN pre-trained model:\u003cbr/\u003e\n[output.zip (google drive)](https://drive.google.com/file/d/11Vu0bL-JaPQUWqHWv1VO89F0WJZWotUm/view?usp=sharing) or \u003cbr /\u003e\n[output.zip (tencent cloud)](https://share.weiyun.com/5nLyD1I)\n\n- unzip the \"output\" folder and put it to *train_tracktor*.\n\n- run training code\n```\npython train_tracktor/experiments/scripts/train_tracktor_full.py\n```\n\nThe trained models are saved by default under *train_tracktor/output/log_full/* folder. \u003cbr /\u003e\nThe tensorboard logs are saved by default under *deepmot/logs/train_log/* folder and you can visualize your training process by:\n```\ntensorboard --logdir=YourGitFolder/train_tracktor/output/log_full/\n```\n**Note:** \n- you should install *tensorflow* (see [tensorflow installation](https://www.tensorflow.org/install/pip)) in order to visualize your training process.\n```\npip install --upgrade tensorflow\n```\n### Train DHN\n- Download the traindata (distance and ground-truth matrices calculated from MOT datasets): \u003cbr/\u003e\n[DHN data (google drive)](https://drive.google.com/file/d/1ICCm6tH_AgPSLzD3qac-6sYOvTIwwTNW/view?usp=sharing) or \u003cbr /\u003e\n[DHN data (tencent cloud)](https://share.weiyun.com/5OKPHxJ)\n\n- unzip DHN_data and put the *DHN_data* folder to *train_DHN/*\n- Run:\n```\npython train_DHN/train_DHN.py --is_cuda --bidirectional\n```\n\nfor more parameter details please run:\n```\npython train_DHN/train_DHN.py -h\n```\nBy default the trained models are saved into *train_DHN/output/DHN/* and log files in *train_DHN/log/*\n\nyour can visualize the training via tensorboard:\n```\ntensorboard --logdir=YourGitFolder/train_DHN/log/\n```\n**Note:** \n- you should install *tensorflow* (see [tensorflow installation](https://www.tensorflow.org/install/pip)) in order to visualize your training process.\n```\npip install --upgrade tensorflow\n```\n\n### Evaluation\nYou can run *test_tracktor/experiments/scripts/evaluate.py* to evaluate your tracker's performance.\n- fill the list *predt_pth* in the code with the folder where the results (.txt files) are saved.\n- make sure the data path is correctly set.\n- then run\n```\npython test_tracktor/experiments/scripts/evaluate.py\n```\n\n### Results\nMOT17 public detections:\n\n|  dataset  | MOTA     | MOTP     | FN     | FP    | IDsW | Total Nb. Objs |\n|-----------|----------|----------|--------|-------|------|----------------|\n|   train   |  62.5%   |  91.7%   | 124786 | 887   | 798  |     336891     |\n|   test    |  53.7%   |  77.2%   | 247447 | 11731 | 1947 |     564228     |\n\nMOT16 public detections:\n\n|  dataset  | MOTA     | MOTP     | FN     | FP    | IDsW | Total Nb. Objs |\n|-----------|----------|----------|--------|-------|------|----------------|\n|   train   |  58.8%   |  92.2%   | 44711  | 538   | 229  |     110407     |\n|   test    |  54.8%   |  77.5%   | 78765  | 2955  | 645  |     182326     |\n\n\nMOT16/17 private detections:\n\n|  dataset  | MOTA     | MOTP     | FN     | FP    | IDsW | Total Nb. Objs |\n|-----------|----------|----------|--------|-------|------|----------------|\n|   train   |  70.0%   |  91.3%   | 32513  | 552   | 677  |     112297     |\n\n\n**Note:** \n- the results can be slightly different depending on the running environment.\n\n\n## Demo \u003ca name=\"demo\"\u003e\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/yihongXU/deepMOT/raw/master/trackingresults.gif\" width=\"500px\" /\u003e\n\u003c/div\u003e\n\n## Acknowledgement \u003ca name=\"Acknowledgement\"\u003e\nSome code is modified and network pre-trained weights are obtained from the following repositories: \u003cbr /\u003e\n\n\n\n**Single Object Tracker**: [**SiamRPN**](https://github.com/foolwood/DaSiamRPN), [**Tracktor**](https://github.com/phil-bergmann/tracking_wo_bnw/tree/iccv_19), [**Faster-RCNN pytorch implementation**](https://github.com/jwyang/faster-rcnn.pytorch/).\n```\n@inproceedings{Zhu_2018_ECCV,\n  title={Distractor-aware Siamese Networks for Visual Object Tracking},\n  author={Zhu, Zheng and Wang, Qiang and Bo, Li and Wu, Wei and Yan, Junjie and Hu, Weiming},\n  booktitle={European Conference on Computer Vision},\n  year={2018}\n}\n\n@InProceedings{Li_2018_CVPR,\n  title = {High Performance Visual Tracking With Siamese Region Proposal Network},\n  author = {Li, Bo and Yan, Junjie and Wu, Wei and Zhu, Zheng and Hu, Xiaolin},\n  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},\n  year = {2018}\n}\n\n@InProceedings{tracktor_2019_ICCV,\n  author = {Bergmann, Philipp and Meinhardt, Tim and Leal{-}Taix{\\'{e}}}, Laura},\n  title = {Tracking Without Bells and Whistles},\n  booktitle = {The IEEE International Conference on Computer Vision (ICCV)},\n  month = {October},\n  year = {2019}}\n\n@inproceedings{10.5555/2969239.2969250,\nauthor = {Ren, Shaoqing and He, Kaiming and Girshick, Ross and Sun, Jian},\ntitle = {Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks},\nyear = {2015},\npublisher = {MIT Press},\naddress = {Cambridge, MA, USA},\nbooktitle = {Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1},\npages = {91–99},\nnumpages = {9},\nlocation = {Montreal, Canada},\nseries = {NIPS’15}\n}\n```\n**MOT Metrics in Python**: [**py-motmetrics**](https://github.com/cheind/py-motmetrics)\u003cbr /\u003e\n**Appearance Features Extractor**: [**DAN**](https://github.com/shijieS/SST)\u003cbr /\u003e\n```\n@article{sun2018deep,\n  title={Deep Affinity Network for Multiple Object Tracking},\n  author={Sun, ShiJie and Akhtar, Naveed and Song, HuanSheng and Mian, Ajmal and Shah, Mubarak},\n  journal={arXiv preprint arXiv:1810.11780},\n  year={2018}\n}\n```\nTraining and testing Data from: \u003cbr /\u003e\n**MOT Challenge**: [**motchallenge**](https://motchallenge.net/data)\n```\n@article{MOT16,\n    title = {{MOT}16: {A} Benchmark for Multi-Object Tracking},\n    shorttitle = {MOT16},\n    url = {http://arxiv.org/abs/1603.00831},\n    journal = {arXiv:1603.00831 [cs]},\n    author = {Milan, A. and Leal-Taix\\'{e}, L. and Reid, I. and Roth, S. and Schindler, K.},\n    month = mar,\n    year = {2016},\n    note = {arXiv: 1603.00831},\n    keywords = {Computer Science - Computer Vision and Pattern Recognition}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FyihongXU%2FdeepMOT","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FyihongXU%2FdeepMOT","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FyihongXU%2FdeepMOT/lists"}