Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/fidler-lab/curve-gcn

Official PyTorch code for Curve-GCN (CVPR 2019)
https://github.com/fidler-lab/curve-gcn

Last synced: 3 months ago
JSON representation

Official PyTorch code for Curve-GCN (CVPR 2019)

Host: GitHub
URL: https://github.com/fidler-lab/curve-gcn
Owner: fidler-lab
License: gpl-3.0
Created: 2019-03-10T19:21:55.000Z (over 5 years ago)
Default Branch: master
Last Pushed: 2020-01-29T20:33:06.000Z (over 4 years ago)
Last Synced: 2024-01-16T19:22:24.417Z (5 months ago)
Homepage:
Size: 246 KB
Stars: 836
Watchers: 39
Forks: 133
Open Issues: 10
Metadata Files:
- Readme: README.md
- License: LICENSE

Lists

awesome-stars - curve-gcn - GCN (CVPR 2019) | fidler-lab | 836 | (Others)
awesome-gcn - fidler-lab/curve-gcn - GCN (CVPR 2019), (Image Segmentation:)
awesome-computer-vision-resources - Curve-GCN

README

# Curve-GCN

This is the official PyTorch implementation of Curve-GCN (CVPR 2019). This repository allows you to train new Curve-GCN models. For technical details, please refer to:
----------------------- ------------------------------------
**Fast Interactive Object Annotation with Curve-GCN**
[Huan Ling](http:///www.cs.toronto.edu/~linghuan/)\* ^1,2, [Jun Gao](http://www.cs.toronto.edu/~jungao/)\* ^1,2, [Amlan Kar](http://www.cs.toronto.edu/~amlan/)^1,2, [Wenzheng Chen](http://www.cs.toronto.edu/~wenzheng/)^1,2, [Sanja Fidler](http://www.cs.toronto.edu/~fidler/)^1,2,3
¹ University of Toronto ² Vector Institute ³ NVIDIA
**[[Paper](https://arxiv.org/pdf/1903.06874.pdf)] [[Video](https://youtu.be/ycD2BtO-QzU)] [[Demo Coming Soon]()] [[Supplementary](http://www.cs.toronto.edu/~linghuan/notes/supplementary_curvegcn.pdf)]**

**CVPR 2019**

*Manually labeling objects by tracing their boundaries is
a laborious process. In Polyrnn, the authors proposed Polygon-
RNN that produces polygonal annotations in a recurrent
manner using a CNN-RNN architecture, allowing interactive
correction via humans-in-the-loop. We propose a new framework
that alleviates the sequential nature of Polygon-RNN,
by predicting all vertices simultaneously using a Graph Convolutional
Network (GCN). Our model is trained end-to-end,
and runs in real time. It supports object annotation by either
polygons or splines, facilitating labeling efficiency for both
line-based and curved objects. We show that Curve-GCN outperforms
all existing approaches in automatic mode, including
the powerful PSP-DeepLab and is significantly
more efficient in interactive mode than Polygon-RNN++.
Our model runs at 29.3ms in automatic, and 2.6ms in interactive
mode, making it 10x and 100x faster than Polygon-
RNN++.*
(\* denotes equal contribution)
----------------------- ------------------------------------

# Where is the code?
To get the code, please [signup](http://www.cs.toronto.edu/annotation/curvegcn/code_signup/) here. We also provide the [dataloader](https://github.com/fidler-lab/curve-gcn/tree/dataloader). We will be using GitHub to keep track of issues with the code and to update on availability of newer versions (also available on website and through e-mail to signed up users).

If you use this code, please cite:

@inproceedings{CurveGCN2019,
title={Fast Interactive Object Annotation with Curve-GCN},
author={Huan Ling and Jun Gao and Amlan Kar and Wenzheng Chen and Sanja Fidler},
booktitle={CVPR},
year={2019}
}

# License

This work is licensed under a *GNU GENERAL PUBLIC LICENSE Version 3* License.

# Environment Setup
All the code has been run and tested on Ubuntu 16.04, Python 2.7.12, Pytorch 0.4.1, CUDA 9.0, TITAN X/Xp and GTX 1080Ti GPUs

- Go into the downloaded code directory
```
cd
```
- Setup python environment
```
virtualenv env
source env/bin/activate
pip install -r requirements.txt
```
- Add the project to PYTHONPATH
```
export PYTHONPATH=$PWD
```

## Data

### Cityscapes
- Download the Cityscapes dataset (leftImg8bit\_trainvaltest.zip) from the official [website](https://www.cityscapes-dataset.com/downloads/) [11 GB]
- Our processed annotation files are included in the download file you get after signing up
- From the root directory, run the following command with appropriate paths to get the annotation files ready for your machine
```
python Scripts/data/change_paths.py --city_dir --json_dir --out_dir
```

## Training

- Download the pre-trained Pytorch Resnet-50 from [here](https://download.pytorch.org/models/resnet50-19c8e357.pth)

### Train Spline GCN

- Modify "exp\_dir", "encoder\_reload", "data\_dir" attributes at Experiments/gnn-active-spline.json
- Run script:
```
python Scripts/train/train_gnn_active_spline.py --exp Experiments/gnn-active-spline.json
```

Checkpoint to reproduce numbers from the paper is available at "checkpoints/Spline_GCN_epoch8_step21000.pth"

### Finetune Spline GCN with Differentiable Rendering
- Modify "exp\_dir", "encoder\_reload", "data\_dir" attributes at Experiments/gnn-active-spline.json
- Modify "xe\_initializer" to be the best checkpoint from the last step.
- Run script:
```
python Scripts/train/train_gnn_active_spline_diffrender.py --exp Experiments/gnn-active-spline-diff-render.json
```

Checkpoint to reproduce numbers from the paper is available at "checkpoints/Spline_GCN_diffrender_epoch6_step18000.pth"

## Prediction
Generate prediction masks:
```
python Scripts/prediction/generate_annotation_from_active_spline.py --exp --output_dir --reload
```

Calculate IOU:
```
python Scripts/get_scores.py --pred --output
```