{"id":13442414,"url":"https://github.com/foolwood/SiamMask","last_synced_at":"2025-03-20T13:33:39.003Z","repository":{"id":38368886,"uuid":"173733017","full_name":"foolwood/SiamMask","owner":"foolwood","description":"[CVPR19/TPAMI23] SiamMask: A Framework for Fast Online Object Tracking and Segmentation","archived":false,"fork":false,"pushed_at":"2025-02-14T03:39:33.000Z","size":7075,"stargazers_count":3484,"open_issues_count":156,"forks_count":810,"subscribers_count":94,"default_branch":"master","last_synced_at":"2025-03-15T01:25:41.233Z","etag":null,"topics":["computer-vision","cvpr2019","deep-learning","object-tracking","pytorch","read-time","video-object-segmentation","visual-tracking"],"latest_commit_sha":null,"homepage":"http://www.robots.ox.ac.uk/~qwang/SiamMask","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/foolwood.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-04T11:34:17.000Z","updated_at":"2025-03-08T23:31:47.000Z","dependencies_parsed_at":"2024-04-20T05:34:03.608Z","dependency_job_id":"0fb482bf-5a94-4852-8e1f-42650ada0a60","html_url":"https://github.com/foolwood/SiamMask","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foolwood%2FSiamMask","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foolwood%2FSiamMask/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foolwood%2FSiamMask/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foolwood%2FSiamMask/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/foolwood","download_url":"https://codeload.github.com/foolwood/SiamMask/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244619330,"owners_count":20482400,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","cvpr2019","deep-learning","object-tracking","pytorch","read-time","video-object-segmentation","visual-tracking"],"created_at":"2024-07-31T03:01:45.439Z","updated_at":"2025-03-20T13:33:38.995Z","avatar_url":"https://github.com/foolwood.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# SiamMask\n\n**NEW:** now including code for both training and inference!\n\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/fast-online-object-tracking-and-segmentation/visual-object-tracking-vot201718)](https://paperswithcode.com/sota/visual-object-tracking-vot201718?p=fast-online-object-tracking-and-segmentation)\n\nThis is the official implementation with *training* code for SiamMask (CVPR2019). For technical details, please refer to:\n\n\n**SiamMask: A Framework for Fast Online Object Tracking and Segmentation** \u003cbr /\u003e\n[Weiming Hu](https://scholar.google.com/citations?user=Wl4tl4QAAAAJ\u0026hl=en), [Qiang Wang](http://www.robots.ox.ac.uk/~qwang/)\\*, [Li Zhang](http://www.robots.ox.ac.uk/~lz)\\*, [Luca Bertinetto](http://www.robots.ox.ac.uk/~luca)\\*, [Philip H.S. Torr](https://scholar.google.it/citations?user=kPxa2w0AAAAJ\u0026hl=en\u0026oi=ao) (\\* denotes equal contribution) \u003cbr /\u003e\n**TPAMI 2023** \u003cbr /\u003e\n**[[Paper](https://ieeexplore.ieee.org/document/10036241)] [[ArXiv](https://arxiv.org/abs/2207.02088)]** \u003cbr /\u003e\n\n\n\n**Fast Online Object Tracking and Segmentation: A Unifying Approach** \u003cbr /\u003e\n[Qiang Wang](http://www.robots.ox.ac.uk/~qwang/)\\*, [Li Zhang](http://www.robots.ox.ac.uk/~lz)\\*, [Luca Bertinetto](http://www.robots.ox.ac.uk/~luca)\\*, [Weiming Hu](https://scholar.google.com/citations?user=Wl4tl4QAAAAJ\u0026hl=en), [Philip H.S. Torr](https://scholar.google.it/citations?user=kPxa2w0AAAAJ\u0026hl=en\u0026oi=ao) (\\* denotes equal contribution) \u003cbr /\u003e\n**CVPR 2019** \u003cbr /\u003e\n**[[Paper](https://arxiv.org/abs/1812.05050)] [[Video](https://youtu.be/I_iOVrcpEBw)] [[Project Page](http://www.robots.ox.ac.uk/~qwang/SiamMask)]** \u003cbr /\u003e\n\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"http://www.robots.ox.ac.uk/~qwang/SiamMask/img/SiamMask.jpg\" width=\"600px\" /\u003e\n\u003c/div\u003e\n\n### Bibtex\nIf you find this code useful, please consider citing:\n\n```\n@article{hu2023siammask,\n  title={Siammask: A framework for fast online object tracking and segmentation},\n  author={Hu, Weiming and Wang, Qiang and Zhang, Li and Bertinetto, Luca and Torr, Philip HS},\n  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},\n  volume={45},\n  number={3},\n  pages={3072--3089},\n  year={2023},\n  publisher={IEEE}\n}\n\n@inproceedings{wang2019fast,\n    title={Fast online object tracking and segmentation: A unifying approach},\n    author={Wang, Qiang and Zhang, Li and Bertinetto, Luca and Hu, Weiming and Torr, Philip HS},\n    booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},\n    year={2019}\n}\n```\n\n\n## Contents\n1. [Environment Setup](#environment-setup)\n2. [Demo](#demo)\n3. [Testing Models](#testing-models)\n4. [Training Models](#training-models)\n\n## Environment setup\nThis code has been tested on Ubuntu 16.04, Python 3.6, Pytorch 0.4.1, CUDA 9.2, RTX 2080 GPUs\n\n- Clone the repository \n```\ngit clone https://github.com/foolwood/SiamMask.git \u0026\u0026 cd SiamMask\nexport SiamMask=$PWD\n```\n- Setup python environment\n```\nconda create -n siammask python=3.6\nsource activate siammask\npip install -r requirements.txt\nbash make.sh\n```\n- Add the project to your PYTHONPATH\n```\nexport PYTHONPATH=$PWD:$PYTHONPATH\n```\n\n## Demo\n- [Setup](#environment-setup) your environment\n- Download the SiamMask model\n```shell\ncd $SiamMask/experiments/siammask_sharp\nwget http://www.robots.ox.ac.uk/~qwang/SiamMask_VOT.pth\nwget http://www.robots.ox.ac.uk/~qwang/SiamMask_DAVIS.pth\n```\n- Run `demo.py`\n\n```shell\ncd $SiamMask/experiments/siammask_sharp\nexport PYTHONPATH=$PWD:$PYTHONPATH\npython ../../tools/demo.py --resume SiamMask_DAVIS.pth --config config_davis.json\n```\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"http://www.robots.ox.ac.uk/~qwang/SiamMask/img/SiamMask_demo.gif\" width=\"500px\" /\u003e\n\u003c/div\u003e\n\n\n## Testing\n- [Setup](#environment-setup) your environment\n- Download test data\n```shell\ncd $SiamMask/data\nsudo apt-get install jq\nbash get_test_data.sh\n```\n- Download pretrained models\n```shell\ncd $SiamMask/experiments/siammask_sharp\nwget http://www.robots.ox.ac.uk/~qwang/SiamMask_VOT.pth\nwget http://www.robots.ox.ac.uk/~qwang/SiamMask_VOT_LD.pth\nwget http://www.robots.ox.ac.uk/~qwang/SiamMask_DAVIS.pth\n```\n- Evaluate performance on [VOT](http://www.votchallenge.net/)\n```shell\nbash test_mask_refine.sh config_vot.json SiamMask_VOT.pth VOT2016 0\nbash test_mask_refine.sh config_vot.json SiamMask_VOT.pth VOT2018 0\nbash test_mask_refine.sh config_vot.json SiamMask_VOT.pth VOT2019 0\nbash test_mask_refine.sh config_vot18.json SiamMask_VOT_LD.pth VOT2016 0\nbash test_mask_refine.sh config_vot18.json SiamMask_VOT_LD.pth VOT2018 0\npython ../../tools/eval.py --dataset VOT2016 --tracker_prefix C --result_dir ./test/VOT2016\npython ../../tools/eval.py --dataset VOT2018 --tracker_prefix C --result_dir ./test/VOT2018\npython ../../tools/eval.py --dataset VOT2019 --tracker_prefix C --result_dir ./test/VOT2019\n```\n- Evaluate performance on [DAVIS](https://davischallenge.org/) (less than 50s)\n```shell\nbash test_mask_refine.sh config_davis.json SiamMask_DAVIS.pth DAVIS2016 0\nbash test_mask_refine.sh config_davis.json SiamMask_DAVIS.pth DAVIS2017 0\n```\n- Evaluate performance on [Youtube-VOS](https://youtube-vos.org/) (need download data from [website](https://youtube-vos.org/dataset/download))\n```shell\nbash test_mask_refine.sh config_davis.json SiamMask_DAVIS.pth ytb_vos 0\n```\n\n### Results\nThese are the reproduction results from this repository. All results can be downloaded from our [project page](http://www.robots.ox.ac.uk/~qwang/SiamMask/).\n\n|                           \u003csub\u003eTracker\u003c/sub\u003e                           |      \u003csub\u003eVOT2016\u003c/br\u003eEAO /  A / R\u003c/sub\u003e     |      \u003csub\u003eVOT2018\u003c/br\u003eEAO / A / R\u003c/sub\u003e      |  \u003csub\u003eDAVIS2016\u003c/br\u003eJ / F\u003c/sub\u003e  |  \u003csub\u003eDAVIS2017\u003c/br\u003eJ / F\u003c/sub\u003e  |     \u003csub\u003eYoutube-VOS\u003c/br\u003eJ_s / J_u / F_s / F_u\u003c/sub\u003e     |     \u003csub\u003eSpeed\u003c/sub\u003e     |\n|:----------------------------------------------------------------------:|:--------------------------------------------:|:--------------------------------------------:|:--------------------------------:|:--------------------------------:|:--------------------------------------------------------:|:------------------------:|\n| \u003csub\u003e[SiamMask-box](http://www.robots.ox.ac.uk/~qwang/SiamMask/)\u003c/sub\u003e |       \u003csub\u003e0.412/0.623/0.233\u003c/sub\u003e       |       \u003csub\u003e0.363/0.584/0.300\u003c/sub\u003e       |               - / -              |               - / -              |                      - / - / - / -                       | \u003csub\u003e**77** FPS\u003c/sub\u003e |\n| \u003csub\u003e[SiamMask](http://www.robots.ox.ac.uk/~qwang/SiamMask/)\u003c/sub\u003e | \u003csub\u003e**0.433**/**0.639**/**0.214**\u003c/sub\u003e | \u003csub\u003e**0.380**/**0.609**/**0.276**\u003c/sub\u003e | \u003csub\u003e**0.713**/**0.674**\u003c/sub\u003e | \u003csub\u003e**0.543**/**0.585**\u003c/sub\u003e | \u003csub\u003e**0.602**/**0.451**/**0.582**/**0.477**\u003c/sub\u003e |   \u003csub\u003e56 FPS\u003c/sub\u003e   |\n| \u003csub\u003e[SiamMask-LD](http://www.robots.ox.ac.uk/~qwang/SiamMask/)\u003c/sub\u003e | \u003csub\u003e**0.455**/**0.634**/**0.219**\u003c/sub\u003e | \u003csub\u003e**0.423**/**0.615**/**0.248**\u003c/sub\u003e | - / - | - / - | - / - / - / - | \u003csub\u003e56 FPS\u003c/sub\u003e |\n\n**Note:** \n- Speed are tested on a NVIDIA RTX 2080. \n- `-box` reports an axis-aligned bounding box from the box branch.\n- `-LD` means training with large dataset (ytb-bb+ytb-vos+vid+coco+det).\n\n\n## Training\n\n### Training Data \n- Download the [Youtube-VOS](https://youtube-vos.org/dataset/download/), \n[COCO](http://cocodataset.org/#download), \n[ImageNet-DET](http://image-net.org/challenges/LSVRC/2015/), \nand [ImageNet-VID](http://image-net.org/challenges/LSVRC/2015/).\n- Preprocess each datasets according the [readme](data/coco/readme.md) files.\n\n### Download the pre-trained model (174 MB)\n(This model was trained on the ImageNet-1k Dataset)\n```\ncd $SiamMask/experiments\nwget http://www.robots.ox.ac.uk/~qwang/resnet.model\nls | grep siam | xargs -I {} cp resnet.model {}\n```\n\n### Training SiamMask base model\n- [Setup](#environment-setup) your environment\n- From the experiment directory, run\n```\ncd $SiamMask/experiments/siammask_base/\nbash run.sh\n```\n- Training takes about 10 hours in our 4 Tesla V100 GPUs.\n- If you experience out-of-memory errors, you can reduce the batch size in `run.sh`.\n- You can view progress on Tensorboard (logs are at \u003cexperiment\\_dir\u003e/logs/)\n- After training, you can test checkpoints on VOT dataset.\n```shell\nbash test_all.sh -s 1 -e 20 -d VOT2018 -g 4  # test all snapshots with 4 GPUs\n```\n- Select best model for hyperparametric search.\n```shell\n#bash test_all.sh -m [best_test_model] -d VOT2018 -n [thread_num] -g [gpu_num] # 8 threads with 4 GPUS\nbash test_all.sh -m snapshot/checkpoint_e12.pth -d VOT2018 -n 8 -g 4 # 8 threads with 4 GPUS\n```\n\n### Training SiamMask model with the Refine module\n- [Setup](#environment-setup) your environment\n- In the experiment file, train with the best SiamMask base model\n```\ncd $SiamMask/experiments/siammask_sharp\nbash run.sh \u003cbest_base_model\u003e\nbash run.sh checkpoint_e12.pth\n```\n- You can view progress on Tensorboard (logs are at \u003cexperiment\\_dir\u003e/logs/)\n- After training, you can test checkpoints on VOT dataset\n```shell\nbash test_all.sh -s 1 -e 20 -d VOT2018 -g 4\n```\n\n### Training SiamRPN++ model (*unofficial*)\n- [Setup](#environment-setup) your environment\n- From the experiment directory, run\n```\ncd $SiamMask/experiments/siamrpn_resnet\nbash run.sh\n```\n- You can view progress on Tensorboard (logs are at \u003cexperiment\\_dir\u003e/logs/)\n- After training, you can test checkpoints on VOT dataset\n```shell\nbash test_all.sh -h\nbash test_all.sh -s 1 -e 20 -d VOT2018 -g 4\n```\n\n## License\nLicensed under an MIT license.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoolwood%2FSiamMask","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffoolwood%2FSiamMask","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoolwood%2FSiamMask/lists"}