{"id":13442144,"url":"https://github.com/hustvl/SparseTrack","last_synced_at":"2025-03-20T13:32:40.123Z","repository":{"id":173848816,"uuid":"650538521","full_name":"hustvl/SparseTrack","owner":"hustvl","description":"Official PyTorch implementation of SparseTrack (the new version of code will come soon)","archived":false,"fork":false,"pushed_at":"2023-12-06T07:22:57.000Z","size":2323,"stargazers_count":132,"open_issues_count":17,"forks_count":12,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-09-11T19:37:17.245Z","etag":null,"topics":["multi-object-tracking","object-detection","object-tracking"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hustvl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-06-07T09:28:03.000Z","updated_at":"2024-09-07T13:44:25.000Z","dependencies_parsed_at":"2023-10-11T11:30:14.455Z","dependency_job_id":"6b138653-9e3c-4209-a001-c5db9ee632c1","html_url":"https://github.com/hustvl/SparseTrack","commit_stats":null,"previous_names":["hustvl/sparsetrack"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hustvl%2FSparseTrack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hustvl%2FSparseTrack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hustvl%2FSparseTrack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hustvl%2FSparseTrack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hustvl","download_url":"https://codeload.github.com/hustvl/SparseTrack/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221768470,"owners_count":16877642,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["multi-object-tracking","object-detection","object-tracking"],"created_at":"2024-07-31T03:01:42.175Z","updated_at":"2025-03-20T13:32:40.115Z","avatar_url":"https://github.com/hustvl.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\n\n# SparseTrack\n####  SparseTrack is a simple and strong multi-object tracker. \n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/sparsetrack-multi-object-tracking-by/multi-object-tracking-on-mot20-1)](https://paperswithcode.com/sota/multi-object-tracking-on-mot20-1?p=sparsetrack-multi-object-tracking-by)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/sparsetrack-multi-object-tracking-by/multi-object-tracking-on-mot17)](https://paperswithcode.com/sota/multi-object-tracking-on-mot17?p=sparsetrack-multi-object-tracking-by)\n\n\u003e [**SparseTrack: Multi-Object Tracking by Performing Scene Decomposition based on Pseudo-Depth**](https://arxiv.org/abs/2306.05238)\n\u003e \n\u003e Zelin Liu, Xinggang Wang, Cheng Wang, Wenyu Liu, Xiang Bai\n\u003e \n\u003e *IEEE TCSVT 2025 \u0026 [arXiv 2306.05238](https://arxiv.org/abs/2306.05238)*\n\n\n## News\n- Add yolov8 detector for tracking, please refer to branch v8.\n- Accept by TCSVT!\n\n## Abstract\nExploring robust and efficient association methods has always been an important issue in multiple-object tracking (MOT). Although existing tracking methods have achieved impressive performance, congestion and frequent occlusions still pose challenging problems in multi-object tracking. We reveal that performing sparse decomposition on dense scenes is a crucial step to enhance the performance of associating occluded targets. To this end, we propose a pseudo-depth estimation method for obtaining the relative depth of targets from 2D images.  Secondly, we design a depth cascading matching (DCM) algorithm, which can use the obtained depth information to convert a dense target set into multiple sparse target subsets and perform data association on these sparse target subsets in order from near to far. By integrating the pseudo-depth method and the DCM strategy into the data association process, we propose a new tracker, called SparseTrack. SparseTrack provides a new perspective for solving the challenging crowded scene MOT problem. Only using IoU matching, SparseTrack achieves comparable performance with the state-of-the-art (SOTA) methods on the MOT17 and MOT20 benchmarks.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"assets/DCM.png\" width=\"500\"/\u003e\u003c/p\u003e \n \n## Tracking performance\n### Results on MOT challenge test set\n| Dataset    | HOTA | MOTA | IDF1 | MT | ML | FP | FN | IDs |\n|------------|-------|-------|------|------|-------|-------|------|------|\n|MOT17       | 65.1 | 81.0 | 80.1 | 54.6% | 14.3% | 23904 | 81927 | 1170 |\n|MOT20       | 63.4 | 78.2 | 77.3 | 69.9% | 9.2%  | 25108 | 86720 | 1116 |\n\n ### Comparison on DanceTrack test set\n|  Method  | HOTA | DetA | AssA | MOTA | IDF1 |\n|------------|-------|-------|------|------|-------|\n| SparseTrack | 55.5 (**+7.8**) | 78.9 (**+7.9**) | 39.1 (**+7.0**) | 91.3 (**+1.7**) | 58.3 (**+4.4**) |\n| ByteTrack  |  47.7 | 71.0 | 32.1 | 89.6 | 53.9 | \n    \n**Notes**: \n- All the inference experiments are performed on 1 NVIDIA GeForce RTX 3090 GPUs. \n- Each experiment uses the **same detector and model weights** as [ByteTrack](https://github.com/ifzhang/ByteTrack) . \n- SparseTrack relies on IoU distance association only and do not use any appearance embedding, learnable motion, and attention components.\n \n## Installation\n#### Dependence\nThis project is an implementation version of [Detectron2](https://github.com/facebookresearch/detectron2) and requires the compilation of [OpenCV](https://opencv.org/), [Boost](https://www.boost.org).\n\n#### Compile GMC(Globle Motion Compensation) module\n\u003estep 1: Downloading [pbcvt](https://github.com/Algomorph/pyboostcvconverter), copy the [python_module.cpp](https://github.com/hustvl/SparseTrack/blob/main/python_module.cpp) to the path **\u003c[pbcvt](https://github.com/Algomorph/pyboostcvconverter)/src/\u003e**.\n\u003e\n\u003estep 2: Adding the relevant OpenCV modules in the pbcvt/CMakeLists.txt file. Here's what you should do: locate the line \"find_package(OpenCV COMPONENTS REQUIRED)\" in the CMakeLists.txt file and replace it with \"find_package(OpenCV COMPONENTS core highgui video videoio videostab REQUIRED)\".\n\u003e\n\u003estep 3: Modifying the compilation path in the Makefile file before compiling pbcvt. The main modifications include updating the following entries：CMAKE_SOURCE_DIR, CMAKE_BINARY_DIR, cmake_progress_start.\n\u003e\n\u003estep 4: Compiling [pbcvt](https://github.com/Algomorph/pyboostcvconverter). For example, you can run the following script:\n\u003e```\n\u003ecmake -DPYTHON_DESIRED_VERSION=3.X  -DPYTHON3_INCLUDE_DIR=/home/lzl/miniconda3/envs/d2/include/python3.9 -DPYTHON3_NUMPY_INCLUDE_DIRS=/home/lzl/miniconda3/envs/d2/lib/python3.9/site-packages/numpy -DPYTHON3_LIBRARY=/home/lzl/miniconda3/envs/d2/lib/libpython3.9.so\n\u003e\n\u003e# and then, running:\n\u003emake \n\u003e```\n\u003e\n\u003estep 5: Please copy the \"pbcvt.xxxxxx.so\" file compiled via [pbcvt](https://github.com/Algomorph/pyboostcvconverter) to the **\u003cROOT/SparseTrack/tracker/\u003e** directory.\n \n#### Install\n```shell\ngit clone https://github.com/hustvl/SparseTrack.git\ncd SparseTrack\npip install -r requirements.txt\npip install Cython  \npip install cython_bbox\n```\n\n## Data preparation\nDownload [MOT17](https://motchallenge.net/), [MOT20](https://motchallenge.net/), [CrowdHuman](https://www.crowdhuman.org/), [Cityperson](https://github.com/Zhongdao/Towards-Realtime-MOT/blob/master/DATASET_ZOO.md), [ETHZ](https://github.com/Zhongdao/Towards-Realtime-MOT/blob/master/DATASET_ZOO.md) and put them under ROOT/ in the following structure:\n```\nROOT\n   |\n   |——————SparseTrack(repo)\n   |           └—————mix\n   |                  └——————mix_17/annotations\n   |                  └——————mix_20/annotations\n   |                  └——————ablation_17/annotations\n   |                  └——————ablation_20/annotations\n   |——————MOT17\n   |        └——————train\n   |        └——————test\n   └——————crowdhuman\n   |         └——————Crowdhuman_train\n   |         └——————Crowdhuman_val\n   |         └——————annotation_train.odgt\n   |         └——————annotation_val.odgt\n   └——————MOT20\n   |        └——————train\n   |        └——————test\n   └——————Citypersons\n   |        └——————images\n   |        └——————labels_with_ids\n   └——————ETHZ\n   |        └——————eth01\n   |        └——————...\n   |        └——————eth07\n   └——————dancetrack\n               └——————train\n               └——————train_seqmap.txt\n               └——————test\n               └——————test_seqmap.txt\n               └——————val\n               └——————val_seqmap.txt\n\n   \n```\nThen, you need to turn the datasets to COCO format and mix different training data:\n```\ncd \u003cROOT\u003e/SparseTrack\npython3 tools/convert_mot17_to_coco.py\npython3 tools/convert_mot20_to_coco.py\npython3 tools/convert_crowdhuman_to_coco.py\npython3 tools/convert_cityperson_to_coco.py\npython3 tools/convert_ethz_to_coco.py\npython3 tools/convert_dance_to_coco.py\n```\nCreating different training mix_data:\n```\ncd \u003cROOT\u003e/SparseTrack\n\n# training on CrowdHuman and MOT17 half train, evaluate on MOT17 half val.\npython3 tools/mix_data_ablation.py\n\n# training on CrowdHuman and MOT20 half train, evaluate on MOT20 half val.\npython3 tools/mix_data_ablation_20.py\n\n# training on MOT17, CrowdHuman, ETHZ, Citypersons, evaluate on MOT17 train.\npython3 tools/mix_data_test_mot17.py\n\n# training on MOT20 and CrowdHuman, evaluate on MOT20 train.\npython3 tools/mix_data_test_mot20.py\n```\n\n## Model zoo\nSee [ByteTrack.model_zoo](https://github.com/ifzhang/ByteTrack#model-zoo). We used the publicly available ByteTrack model zoo trained on MOT17, MOT20 and ablation study for YOLOX object detection.\n\nAdditionally, we conducted joint training on MOT20 train half and Crowdhuman, and evaluated on MOT20 val half. The model as follows: [yolox_x_mot20_ablation](https://drive.google.com/file/d/1F2XwyYKj1kefLPUFRHxgnpaAmEwyoocw/view?usp=drive_link)\n\nThe model trained on DanceTrack can be available at google:[yolox_x_dancetrack](https://drive.google.com/drive/folders/1-uxcNTi7dhuDNGC5MmzXyllLzmVbzXay?usp=sharing) or baidu: [yolox_x_dancetrack](https://pan.baidu.com/s/1FIIy9mKnNQQrI7ACCAKRjQ), the extracted key as: sptk\n\n\n## Training\nAll training is conducted on a unified script. You need to change the **VAL_JSON** and **VAL_PATH** in [register_data.py](https://github.com/hustvl/SparseTrack/blob/main/register_data.py), and then run as follows：\n```\n# training on MOT17, CrowdHuman, ETHZ, Citypersons, evaluate on MOT17 train set.\nCUDA_VISIBLE_DEVICES=0,1,2,3 python train.py --num-gpus 4  --config-file mot17_train_config.py \n\n\n# training on MOT20, CrowdHuman, evaluate on MOT20 train set.\nCUDA_VISIBLE_DEVICES=0,1,2,3 python train.py --num-gpus 4  --config-file mot20_train_config.py \n```\n**Notes**: \nFor MOT20, you need to clip the bounding boxes inside the image.\n\nAdd clip operation in line 138-139 in [data_augment.py](https://github.com/hustvl/SparseTrack/blob/main/datasets/data/data_augment.py), line 118-121 in [mosaicdetection.py](https://github.com/hustvl/SparseTrack/blob/main/datasets/data/datasets/mosaicdetection.py), line 213-221 in mosaicdetection.py, line 115-118 in [boxes.py](https://github.com/hustvl/SparseTrack/blob/main/utils/boxes.py).\n\n## Tracking\nAll tracking experimental scripts are run in the following manner. You first place the model weights in the **\u003cROOT/SparseTrack/pretrain/\u003e**, and change the **VAL_JSON** and **VAL_PATH** in [register_data.py](https://github.com/hustvl/SparseTrack/blob/main/register_data.py).\n```\n# tracking on mot17 train set or test set\nCUDA_VISIBLE_DEVICES=0 python3 track.py  --num-gpus 1  --config-file mot17_track_cfg.py \n\n\n# tracking on mot20 train set or test set\nCUDA_VISIBLE_DEVICES=0 python3 track.py  --num-gpus 1  --config-file mot20_track_cfg.py \n\n\n# tracking on mot17 val_half set\nCUDA_VISIBLE_DEVICES=0 python3 track.py  --num-gpus 1  --config-file mot17_ab_track_cfg.py \n\n\n# tracking on mot20 val_half set\nCUDA_VISIBLE_DEVICES=0 python3 track.py  --num-gpus 1  --config-file mot20_ab_track_cfg.py\n```\n\n#### Tracking on dancetrack test set\n\u003e Step 1: Please comment out line 368-373 in the [sparse_tracker.py](https://github.com/hustvl/SparseTrack/blob/main/tracker/sparse_tracker.py) and modify the threshold for low-score matching stage from 0.3 to 0.35 (at line 402 in the sparse_tracker.py).\n\u003e\n\u003e Step 2: Running:\n```\nCUDA_VISIBLE_DEVICES=0 python3 track.py  --num-gpus 1  --config-file dancetrack_sparse_cfg.py\n```\n\n\n \n## Citation\nIf you find SparseTrack is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.\n```bibtex\n@article{liu2025sparsetrack,\n  title={Sparsetrack: Multi-object tracking by performing scene decomposition based on pseudo-depth},\n  author={Liu, Zelin and Wang, Xinggang and Wang, Cheng and Liu, Wenyu and Bai, Xiang},\n  journal={IEEE Transactions on Circuits and Systems for Video Technology},\n  year={2025},\n  publisher={IEEE}\n}\n```\n\n## Acknowledgements\nA large part of the code is borrowed from [YOLOX](https://github.com/Megvii-BaseDetection/YOLOX), [FairMOT](https://github.com/ifzhang/FairMOT), [ByteTrack](https://github.com/ifzhang/ByteTrack), [BoT-SORT](https://github.com/NirAharon/BOT-SORT), [Detectron2](https://github.com/facebookresearch/detectron2). \n Many thanks for their wonderful work.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhustvl%2FSparseTrack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhustvl%2FSparseTrack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhustvl%2FSparseTrack/lists"}