{"id":13443504,"url":"https://github.com/vision4robotics/TCTrack","last_synced_at":"2025-03-20T16:31:55.393Z","repository":{"id":40612965,"uuid":"464366456","full_name":"vision4robotics/TCTrack","owner":"vision4robotics","description":"TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022) \u0026 TCTrack++ (TPAMI)","archived":false,"fork":false,"pushed_at":"2023-08-29T14:42:51.000Z","size":69006,"stargazers_count":165,"open_issues_count":8,"forks_count":34,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-08-01T03:43:47.553Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vision4robotics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-02-28T06:35:24.000Z","updated_at":"2024-07-31T13:18:22.000Z","dependencies_parsed_at":"2024-01-14T15:32:52.926Z","dependency_job_id":null,"html_url":"https://github.com/vision4robotics/TCTrack","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vision4robotics%2FTCTrack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vision4robotics%2FTCTrack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vision4robotics%2FTCTrack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vision4robotics%2FTCTrack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vision4robotics","download_url":"https://codeload.github.com/vision4robotics/TCTrack/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221780031,"owners_count":16879040,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:02:02.339Z","updated_at":"2025-03-20T16:31:55.387Z","avatar_url":"https://github.com/vision4robotics.png","language":"Python","funding_links":[],"categories":["Python","Frameworks"],"sub_categories":[],"readme":"# TCTrack: Temporal Contexts for Aerial Tracking （CVPR2022) \u0026 TCTrack++：Towards Real-World Visual Tracking with Temporal Contexts （TPAMI）\n\n[Ziang Cao](https://ziangcao0312.github.io/) and [Ziyuan Huang](https://huang-ziyuan.github.io/) and [Liang Pan](https://scholar.google.com/citations?user=lSDISOcAAAAJ\u0026hl=zh-CN\u0026authuser=1) and Shiwei Zhang and [Ziwei Liu](https://liuziwei7.github.io/) and [Changhong Fu](https://vision4robotics.github.io/authors/changhong-fu/)\n\nIn CVPR, 2022.\n\n[[paper](https://arxiv.org/abs/2203.01885)] [[paper_journal](https://arxiv.org/abs/2308.10330)]\n\n## Abstract\nTemporal contexts among consecutive frames are far\nfrom being fully utilized in existing visual trackers. In this work, we present TCTrack, a comprehensive framework to fully exploit temporal contexts for aerial tracking. The temporal contexts are incorporated at two levels: the extraction of features and the refinement of similarity maps. Specifically, for feature extraction, an online temporally adaptive convolution is proposed to enhance the spatial features using temporal information, which is achieved by dynamically calibrating the convolution weights according to the previous frames. For similarity map refinement, we propose an adaptive temporal transformer, which first effectively encodes\ntemporal knowledge in a memory-efficient way, before\nthe temporal knowledge is decoded for accurate adjustment\nof the similarity map. TCTrack is effective and efficient:\nevaluation on four aerial tracking benchmarks shows\nits impressive performance; real-world UAV tests show its\nhigh speed of over 27 FPS on NVIDIA Jetson AGX Xavier.\n\n![Workflow of our tracker](https://github.com/vision4robotics/TCTrack/blob/main/images/workflow.jpg)\n\n\nThe implementation of our online temporally adaptive convolution is based on [TadaConv](https://github.com/alibaba-mmai-research/TAdaConv) (ICLR2022).\n\n\n## 1. Environment setup\nThis code has been tested on Ubuntu 18.04, Python 3.8.3, Pytorch 0.7.0/1.6.0, CUDA 10.2.\nPlease install related libraries before running this code: \n```bash\npip install -r requirements.txt\n```\n\n## 2. Test\n\n### (a) TCTrack\nDownload pretrained model by [Baidu](https://pan.baidu.com/s/1jSAcHY9OfarVlxKjOCrVEw) （code: 2u1l) or [Googledrive](https://drive.google.com/file/d/1nWRfvAEcSduR9A4W5MpyZBjp0SCjvmNk/view?usp=sharing) and put it into `tools/snapshot` directory.\n\nDownload testing datasets and put them into `test_dataset` directory. \n\n```bash \npython ./tools/test.py                                \n\t--dataset OTB100                  \n    --tracker_name TCTrack\n\t--snapshot snapshot/general_model.pth # pre-train model path\n```\nThe testing result will be saved in the `results/dataset_name/tracker_name` directory.\n\n**Note:** The results of TCTrack can be [downloaded](https://pan.baidu.com/s/1-V4JbKvmVPm0aOKWTOQtyQ) (code:kh3e).\n\n### (b) TCTrack++\nDownload pretrained model by [baidu](https://pan.baidu.com/s/1aggubJ4F-YdMtEo7t0lYtw?pwd=dj2u) (code:dj2u) [Googledrive](https://drive.google.com/file/d/1yHLZTPkU_Mko0OX03fd2HH01g0gflusI/view?usp=sharing) and put it into `tools/snapshot` directory.\n\nDownload testing datasets and put them into `test_dataset` directory. \n\n```bash \npython ./tools/test.py     # offline evaluation                       \n\t--dataset OTB100                  \n    --tracker_name TCTrack++\n\t--snapshot snapshot/general_model.pth # pre-train model path\n\n```\n```bash \npython ./tools/test_rt.py     # online evaluation                       \n\t--dataset OTB100                  \n    --tracker_name TCTrack++\n\t--snapshot snapshot/general_model.pth # pre-train model path\n```\n\nThe testing result will be saved in the `results/dataset_name/tracker_name` directory.\n\n**Note:** The results of TCTrack++ can be [downloaded](https://drive.google.com/file/d/1TaolHsyOy_zIkm-MEEkMZuOtbr_NuUYC/view?usp=sharing) or [downloaded](https://pan.baidu.com/s/1v7ie10TmFDiWKoosTESXTw?pwd=3vyx) (code: 3vyx).\n\n## 3. Train\n\n### (a) TCTrack\n\n#### Prepare training datasets\n\nDownload the datasets：\n* [VID](http://image-net.org/challenges/LSVRC/2017/)\n* [Lasot](https://paperswithcode.com/dataset/lasot)\n* [GOT-10K](http://got-10k.aitestunion.com/downloads)\n\n\n**Note:** `train_dataset/dataset_name/readme.md` has listed detailed operations about how to generate training datasets.\n\n#### Train a model\nTo train the TCTrack and TCTrack++ model, run `train.py` with the desired configs:\n\n```bash\ncd tools\npython train_tctrack.py\n```\n\n### (b) TCTrack++\n\n#### Prepare training datasets\n\nDownload the datasets：\n* [VID](http://image-net.org/challenges/LSVRC/2017/)\n* [Lasot](https://paperswithcode.com/dataset/lasot)\n* [GOT-10K](http://got-10k.aitestunion.com/downloads)\n* [COCO](http://cocodataset.org)\n\n**Note:** `train_dataset/dataset_name/readme.md` has listed detailed operations about how to generate training datasets.\n\n### Train a model\nTo train the TCTrack and TCTrack++ model, run `train.py` with the desired configs:\n\n```bash\ncd tools\npython train_tctrackpp.py\n```\n\n## 4. Offline Evaluation\nIf you want to evaluate the results of our tracker, please put those results into  `results` directory.\n```\npython eval.py \t                          \\\n\t--tracker_path ./results          \\ # result path\n\t--dataset OTB100                  \\ # dataset_name\n\t--tracker_prefix 'general_model'   # tracker_name\n```\n\n## 5. Online Evaluation\nIf you want to evaluate the results of our tracker, please put the pkl files into  `results_rt_raw` directory.\n\n\n```\n#first step\n\npython rt_eva.py \t                          \\\n\t--raw_root ./tools/results_rt_raw/OTB100          \\ # pkl path\n\t--tar_root ./tools/results_rt/OTB100                  \\ # output txt files for evaluation\n\t--gtroot ./test_dataset/OTB100   # groundtruth of dataset\n```\n\n```\n# second step\npython eval.py \t                          \\\n\t--tracker_path ./results_rt          \\ # result path\n\t--dataset OTB100                  \\ # dataset_name\n\t--trackers TCTrack++   # tracker_name\n```\n\n\n**Note:** The code is implemented based on [pysot-toolkit](https://github.com/StrangerZhang/pysot-toolkit). We would like to express our sincere thanks to the contributors.\n\n## Demo video\n[![TCTrack](https://res.cloudinary.com/marcomontalbano/image/upload/v1646040242/video_to_markdown/images/youtube--wcR3iCFJN4E-c05b58ac6eb4c4700831b2b3070cd403.jpg)](https://youtu.be/wcR3iCFJN4E \"TCTrack\")\n\n## References \n\n```\n@inproceedings{cao2022tctrack,\n\ttitle={{TCTrack: Temporal Contexts for Aerial Tracking}},\n\tauthor={Cao, Ziang and Huang, Ziyuan and Pan, Liang and Zhang, Shiwei and Liu, Ziwei and Fu, Changhong},\n\tbooktitle={CVPR},\n\tpages={14798--14808},\n\tyear={2022}\n}\n\n@article{cao2023towards,\n  title={Towards Real-World Visual Tracking with Temporal Contexts},\n  author={Cao, Ziang and Huang, Ziyuan and Pan, Liang and Zhang, Shiwei and Liu, Ziwei and Fu, Changhong},\n  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},\n  year={2023},\n  publisher={IEEE}\n}\n\n```\n\n## Acknowledgement\nThe code is implemented based on [pysot](https://github.com/STVIR/pysot). We would like to express our sincere thanks to the contributors.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvision4robotics%2FTCTrack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvision4robotics%2FTCTrack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvision4robotics%2FTCTrack/lists"}