{"id":33053202,"url":"https://github.com/Event-AHU/COESOT","last_synced_at":"2025-11-23T15:00:42.783Z","repository":{"id":136861912,"uuid":"564353347","full_name":"Event-AHU/COESOT","owner":"Event-AHU","description":"A large-scale benchmark dataset for color-event based visual tracking","archived":false,"fork":false,"pushed_at":"2025-04-29T14:03:27.000Z","size":41037,"stargazers_count":59,"open_issues_count":1,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-29T15:22:57.358Z","etag":null,"topics":["benchmark-dataset","coesot","dynamic-vision-sensors","event-camera","multi-modal","multi-modality-tracking","rgb-event","single-object-tracking","transformer","visual-object-tracking"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Event-AHU.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-11-10T14:36:33.000Z","updated_at":"2025-04-29T14:03:38.000Z","dependencies_parsed_at":"2025-03-07T14:35:17.361Z","dependency_job_id":null,"html_url":"https://github.com/Event-AHU/COESOT","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Event-AHU/COESOT","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Event-AHU%2FCOESOT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Event-AHU%2FCOESOT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Event-AHU%2FCOESOT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Event-AHU%2FCOESOT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Event-AHU","download_url":"https://codeload.github.com/Event-AHU/COESOT/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Event-AHU%2FCOESOT/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":285968540,"owners_count":27262718,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-23T02:00:06.149Z","response_time":135,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark-dataset","coesot","dynamic-vision-sensors","event-camera","multi-modal","multi-modality-tracking","rgb-event","single-object-tracking","transformer","visual-object-tracking"],"created_at":"2025-11-14T03:00:36.947Z","updated_at":"2025-11-23T15:00:42.775Z","avatar_url":"https://github.com/Event-AHU.png","language":"Python","funding_links":[],"categories":[":punch: :Datasets and Benchmarks","RGBE Tracking"],"sub_categories":["RGBE Datasets","Papers"],"readme":"\u003cdiv align=\"center\"\u003e\n \n\u003cimg src=\"https://github.com/Event-AHU/COESOT/blob/main/figures/COESOT.png\" width=\"600\"\u003e \n\n**A general and large-scale benchmark COESOT dataset for color-event based visual tracking**\n\n ------\n \n\u003c/div\u003e\n\n\u003e **[Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric](https://arxiv.org/abs/2211.11010)**, Chuanming Tang, Xiao Wang, Ju Huang, Bo Jiang, Lin Zhu, Jianlin Zhang, Yaowei Wang, Yonghong Tian \n[[Project](https://sites.google.com/view/coesot/)]\n\n\n### Update Log \n\n\n* :fire: [2025.11.05] COESOT is accepted by the Journal Pattern Recognition!  \n\n* :fire: [2024.03.12] A New Long-term RGB-Event based Visual Object Tracking Benchmark Dataset (termed **FELT**) is available at\n  [[Paper](https://arxiv.org/pdf/2403.05839.pdf)] \n  [[Code](https://github.com/Event-AHU/FELT_SOT_Benchmark)] \n  [[DemoVideo](https://youtu.be/6zxiBHTqOhE?si=6ARRGFdBLSxyp3G8)]\n\n* :fire: [2024.03.06] Tracking results of CEUTrack on **VisEvent** dataset is available at [[ceutrack_visevent_dataset_tracking_results.zip](https://github.com/Event-AHU/COESOT/blob/main/ceutrack_visevent_dataset_tracking_results.zip)] \n\n* :fire: [2023.09.27] A High Definition (HD) Event based Visual Object Tracking Benchmark Dataset (termed **EventVOT**) is available at\n[[arXiv](https://arxiv.org/abs/2309.14611)] [[Github](https://github.com/Event-AHU/EventVOT_Benchmark)] \n\n\n### Demo Video: \n* [[YouTube](https://youtu.be/_ROv09rvi2k)]\n\n\n\n### Dataset Download: \n```\nBaidu Download link：https://pan.baidu.com/s/12XDlKABlz3lDkJJEDvsu9A     Passcode：AHUT \n```\n\nThe directory should have the below format:\n```Shell\n├── COESOT dataset\n    ├── Training Subset (827 videos, 160GB)\n        ├── dvSave-2021_09_01_06_59_10\n            ├── dvSave-2021_09_01_06_59_10_aps\n            ├── dvSave-2021_09_01_06_59_10_dvs\n            ├── dvSave-2021_09_01_06_59_10.aedat4\n            ├── groundtruth.txt\n            ├── absent.txt\n            ├── start_end_index.txt\n        ├── ... \n    ├── Testing Subset (528 videos, 105GB)\n        ├── dvSave-2021_07_30_11_04_12\n            ├── dvSave-2021_07_30_11_04_12_aps\n            ├── dvSave-2021_07_30_11_04_12_dvs\n            ├── dvSave-2021_07_30_11_04_12.aedat4\n            ├── groundtruth.txt\n            ├── absent.txt\n            ├── start_end_index.txt\n        ├── ... \n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"85%\" src=\"https://github.com/Event-AHU/COESOT/blob/main/figures/CODSOT_benchmarkSamples.jpg\" alt=\"Framework\"/\u003e\n\u003c/p\u003e\n\n\n### COESOT_eval_toolkit\n1. unzip the COESOT_eval_toolkit.zip, and open it with Matlab (over Matlab R2020).\n2. add your tracking results and [baseline results (Passcode：siaw)](https://pan.baidu.com/s/1YN07LHERxO31zflMUzgK4A)  in `$/coesot_tracking_results/` and modify the name in `$/utils/config_tracker.m`.    BTW, here we also provide the event-only baseline tracking methods results in [[Event_only Results](https://pan.baidu.com/s/1-8dKCOqt7xtJcoyb8D3RmQ )] Passcode：qblp\n\n\n\n3. run `Evaluate_COESOT_benchmark_SP_PR_only.m` for the overall performance evaluation, including SR, PR, NPR.\n\n\u003cp align=\"left\"\u003e\n  \u003cimg width=\"100%\" src=\"./figures/SRPRNPR.png\" alt=\"SR_PR_NPR\"/\u003e\n\u003c/p\u003e\n\n4. run `plot_BOC.m` for BOC score evaluation and figure plot.\n5. run `plot_radar.m` for attributes radar figrue plot.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"43%\" src=\"./figures/radar1.png\" alt=\"Radar\"/\u003e\u003cimg width=\"57%\" src=\"./figures/BOC_score.png\" alt=\"Radar\"/\u003e\n\u003c/p\u003e\n\n6. run `Evaluate_COESOT_benchmark_attributes.m` for attributes analysis and figure saved in `$/res_fig/`. \n\n\n\n\n# CEUTrack\nA unified framework for color-event tracking. \n\n[[Models](https://pan.baidu.com/s/1B6VPTqoltVCgOCfceK7bTA )] Passcode：0uk0\n[[Raw Results](https://pan.baidu.com/s/1tzLABOFTpF1SNytj05dFzg)] Passcode：yeow\n[[Training logs](https://pan.baidu.com/s/12KHyJZ-X4UQu0xjsoKEPqg )] Passcode：hnim\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"85%\" src=\"./figures/frameworkV2.jpg\" alt=\"Framework\"/\u003e\n\u003c/p\u003e\n\n\nInstall env\n```\nconda create -n event python=3.7\nconda activate event\nbash install.sh\n```\n\nRun the following command to set paths for this project\n```\npython tracking/create_default_local_file.py --workspace_dir . --data_dir ./data --save_dir ./output\n```\n\nAfter running this command, you can also modify paths by editing these two files\n```\nlib/train/admin/local.py  # paths about training\nlib/test/evaluation/local.py  # paths about testing\n```\n\nThen, put the tracking datasets COESOT in `./data`. \n\nDownload pre-trained [MAE ViT-Base weights](https://dl.fbaipublicfiles.com/mae/pretrain/mae_pretrain_vit_base.pth) and put it under `$/pretrained_models`\n\nDownload the model weights and put it on `$/output/checkpoints/train/ceutrack`\n\n\n\n* **[Note] More useful scripts can be found from:** \n```\nhttps://github.com/Event-AHU/COESOT/tree/main/CEUTrack/scripts\n```\n\n\n## Train \u0026 Test \u0026 Evaluation\n```\n    # train\n    export CUDA_VISIBLE_DEVICES=0\n    python tracking/train.py --script ceutrack --config ceutrack_coesot  \\\n    --save_dir ./output --mode multiple --nproc_per_node 1 --use_wandb  0\n    # test\n    python tracking/test.py   ceutrack ceutrack_coesot --dataset coesot --threads 4 --num_gpus 1\n    # eval\n    python tracking/analysis_results.py --dataset coesot  --parameter_name ceutrack_coesot\n```\n\n\n\n\n### Test FLOPs, and Speed\n*Note:* The speeds reported in our paper were tested on a single RTX 3090 GPU.\n\n```\n# Profiling ceutrack_coesot\npython tracking/profile_model.py --script ceutrack --config ceutrack_coesot\n```\n\n\n### Activation Visualization \nUse the script from: [[show_CAM.py](https://github.com/Event-AHU/COESOT/blob/main/CEUTrack/scripts/show_CAM.py)]\n\n```\nfrom .show_CAM import getCAM\ngetCAM(response, curr_image, self.idx)\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"85%\" src=\"./figures/responseMAPs.png\" alt=\"responseMAPs\"/\u003e\n\u003c/p\u003e\n\n\n\n\n\n## TODO List \n- [x] Paper (arXiv) release\n- [x] COESOT dataset release\n- [x] Evaluation Toolkit release\n- [x] Source Code release\n- [x] Tracking Models release\n\n\n\n### Acknowledgments\n* Thanks for the [OSTrack](https://github.com/botaoye/OSTrack), [PyTracking](https://github.com/visionml/pytracking) and [ViT](https://github.com/rwightman/pytorch-image-models) library for a quickly implement.\n\n\n### Citation: \n```bibtex\n@article{tang2022coesot,\n  title={Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric},\n  author={Tang, Chuanming and Wang, Xiao and Huang, Ju and Jiang, Bo and Zhu, Lin and Zhang, Jianlin and Wang, Yaowei and Tian, Yonghong},\n  journal={arXiv preprint arXiv:2211.11010},\n  year={2022}\n}\n```\n\n\n\n## Star History\n\n\u003ca href=\"https://star-history.com/#Event-AHU/COESOT\u0026Date\"\u003e\n \u003cpicture\u003e\n   \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://api.star-history.com/svg?repos=Event-AHU/COESOT\u0026type=Date\u0026theme=dark\" /\u003e\n   \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"https://api.star-history.com/svg?repos=Event-AHU/COESOT\u0026type=Date\" /\u003e\n   \u003cimg alt=\"Star History Chart\" src=\"https://api.star-history.com/svg?repos=Event-AHU/COESOT\u0026type=Date\" /\u003e\n \u003c/picture\u003e\n\u003c/a\u003e\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEvent-AHU%2FCOESOT","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FEvent-AHU%2FCOESOT","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEvent-AHU%2FCOESOT/lists"}