https://github.com/carrierlxk/COSNet

See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks (CVPR19)
https://github.com/carrierlxk/COSNet

attention-siamese-networks co-attention cvpr2019 object-segmentation segmentation video-object-segmentation video-segmentation

Last synced: 2 months ago
JSON representation

See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks (CVPR19)

• Host: GitHub
• URL: https://github.com/carrierlxk/COSNet
• Owner: carrierlxk
• Created: 2019-03-28T19:11:59.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2021-02-25T13:06:54.000Z (over 3 years ago)
• Last Synced: 2024-06-07T22:38:20.213Z (4 months ago)
• Topics: attention-siamese-networks, co-attention, cvpr2019, object-segmentation, segmentation, video-object-segmentation, video-segmentation
• Language: Python
• Homepage:
• Size: 1.42 MB
• Stars: 320
• Watchers: 11
• Forks: 62
• Open Issues: 16
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-co-saliency-detection - Code

README

        
# COSNet

Code for CVPR 2019 paper: 

[See More, Know More: Unsupervised Video Object Segmentation with

Co-Attention Siamese Networks](http://openaccess.thecvf.com/content_CVPR_2019/papers/Lu_See_More_Know_More_Unsupervised_Video_Object_Segmentation_With_Co-Attention_CVPR_2019_paper.pdf)

[Xiankai Lu](https://sites.google.com/site/xiankailu111/), [Wenguan Wang](https://sites.google.com/view/wenguanwang), Chao Ma, Jianbing Shen, Ling Shao, Fatih Porikli

##

![](../master/framework.png)

- - -

:new:

Our group co-attention achieves a further performance gain (81.1 mean J on DAVIS-16 dataset), related codes have also been released.

The pre-trained model, testing and training code:

### Quick Start

#### Testing

1. Install pytorch (version:1.0.1).

2. Download the pretrained model. Run 'test_coattention_conf.py' and change the davis dataset path, pretrainde model path and result path.

3. Run command: python test_coattention_conf.py --dataset davis --gpus 0

4. Post CRF processing code comes from: https://github.com/lucasb-eyer/pydensecrf. 

The pretrained weight can be download from [GoogleDrive](https://drive.google.com/open?id=14ya3ZkneeHsegCgDrvkuFtGoAfVRgErz) or [BaiduPan](https://pan.baidu.com/s/16oFzRmn4Meuq83fCYr4boQ), pass code: xwup.

The segmentation results on DAVIS, FBMS and Youtube-objects can be download from DAVIS_benchmark(https://davischallenge.org/davis2016/soa_compare.html) or

[GoogleDrive](https://drive.google.com/open?id=1JRPc2kZmzx0b7WLjxTPD-kdgFdXh5gBq) or [BaiduPan](https://pan.baidu.com/s/11n7zAt3Lo2P3-42M2lsw6Q), pass code: q37f.

The youtube-objects dataset can be downloaded from [here](http://calvin-vision.net/datasets/youtube-objects-dataset/) and annotation can be found [here](http://vision.cs.utexas.edu/projects/videoseg/data_download_register.html).

The FBMS dataset can be downloaded from [here](https://lmb.informatik.uni-freiburg.de/resources/datasets/moseg.en.html).

#### Training

1. Download all the training datasets, including MARA10K and DUT saliency datasets. Create a folder called images and put these two datasets into the folder. 

2. Download the deeplabv3 model from [GoogleDrive](https://drive.google.com/open?id=1hy0-BAEestT9H4a3Sv78xrHrzmZga9mj). Put it into the folder pretrained/deep_labv3.

3. Change the video path, image path and deeplabv3 path in train_iteration_conf.py.  Create two txt files which store the saliency dataset name and DAVIS16 training sequences name. Change the txt path in PairwiseImg_video.py.

4. Run command: python train_iteration_conf.py --dataset davis --gpus 0,1

### Citation

If you find the code and dataset useful in your research, please consider citing:

```

@InProceedings{Lu_2019_CVPR,  

author = {Lu, Xiankai and Wang, Wenguan and Ma, Chao and Shen, Jianbing and Shao, Ling and Porikli, Fatih},  

title = {See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks},  

booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},  

year = {2019}  

}

@article{lu2020_pami,

  title={Zero-Shot Video Object Segmentation with Co-Attention Siamese Networks},

  author={Lu, Xiankai and Wang, Wenguan and Shen, Jianbing and Crandall, David and Luo, Jiebo},

  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},

  year={2020},

  publisher={IEEE}

}

```

### Other related projects/papers:

[Saliency-Aware Geodesic Video Object Segmentation (CVPR15)](https://github.com/wenguanwang/saliencysegment)

[Learning Unsupervised Video Primary Object Segmentation through Visual Attention (CVPR19)](https://github.com/wenguanwang/AGS)

Any comments, please email: [email protected]