{"id":13935353,"url":"https://github.com/fidler-lab/polyrnn-pp-pytorch","last_synced_at":"2025-04-05T01:08:41.325Z","repository":{"id":44036084,"uuid":"144767237","full_name":"fidler-lab/polyrnn-pp-pytorch","owner":"fidler-lab","description":"PyTorch training/tool code for Polygon-RNN++ (CVPR 2018)","archived":false,"fork":false,"pushed_at":"2019-07-24T19:49:56.000Z","size":990,"stargazers_count":702,"open_issues_count":17,"forks_count":107,"subscribers_count":33,"default_branch":"master","last_synced_at":"2025-03-29T00:11:53.891Z","etag":null,"topics":["annotation","cvpr2018","deep-learning","instance-segmentation","labelling","polygon-rnn","pytorch"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fidler-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-08-14T20:07:46.000Z","updated_at":"2025-02-27T02:26:56.000Z","dependencies_parsed_at":"2022-09-08T01:20:58.103Z","dependency_job_id":null,"html_url":"https://github.com/fidler-lab/polyrnn-pp-pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fidler-lab%2Fpolyrnn-pp-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fidler-lab%2Fpolyrnn-pp-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fidler-lab%2Fpolyrnn-pp-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fidler-lab%2Fpolyrnn-pp-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fidler-lab","download_url":"https://codeload.github.com/fidler-lab/polyrnn-pp-pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247271532,"owners_count":20911587,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["annotation","cvpr2018","deep-learning","instance-segmentation","labelling","polygon-rnn","pytorch"],"created_at":"2024-08-07T23:01:38.493Z","updated_at":"2025-04-05T01:08:41.302Z","avatar_url":"https://github.com/fidler-lab.png","language":null,"funding_links":[],"categories":["Others"],"sub_categories":[],"readme":"# PolygonRNN++\n\nThis is the official PyTorch reimplementation of Polygon-RNN++ (CVPR 2018). This repository allows you to train new Polygon-RNN++ models, and run our demo tool on local machines. For technical details, please refer to:\n\n**Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++**  \n[David Acuna](http://www.cs.toronto.edu/~davidj/)\\*, [Huan Ling](http:///www.cs.toronto.edu/~linghuan/)\\*, [Amlan Kar](http://www.cs.toronto.edu/~amlan/)\\*, [Sanja Fidler](http://www.cs.toronto.edu/~fidler/) (\\* denotes equal contribution)   \nCVPR 2018  \n**[[Paper](https://arxiv.org/abs/1803.09693)] [[Video](https://www.youtube.com/watch?v=evGqMnL4P3E)] [[Project Page](http://www.cs.toronto.edu/polyrnn/)] [[Demo](http://www.cs.toronto.edu/~amlan/demo/)]**  \n\u003cimg src = \"Docs/model.png\" width=\"56%\"/\u003e\n\u003cimg src = \"Docs/polydemo.gif\" width=\"42%\"/\u003e\n\n# Where is the code?\nTo get the code, please [signup](http://www.cs.toronto.edu/polyrnn/code_signup/) here. We will be using GitHub to keep track of issues with the code and to update on availability of newer versions (also available on website and through e-mail to signed up users).\n\nIf you use this code, please cite:\n\n    @inproceedings{AcunaCVPR18,\n    title={Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++},\n    author={David Acuna and Huan Ling and Amlan Kar and Sanja Fidler},\n    booktitle={CVPR},\n    year={2018}\n    }\n\n    @inproceedings{CastrejonCVPR17,\n    title = {Annotating Object Instances with a Polygon-RNN},\n    author = {Lluis Castrejon and Kaustav Kundu and Raquel Urtasun and Sanja Fidler},\n    booktitle = {CVPR},\n    year = {2017}\n    }\n\n# Contents\n1. [Reproduction Results](#results)\n2. [Environment Setup](#environment-setup)\n3. [Tool](#tool)\n    1. [Backend](#backend)\n    2. [Frontend](#frontend)\n4. [Testing Models](#testing-models)\n5. [Training Models](#training-models)\n    1. [Data](#data)\n    2. [Training MLE Model](#training-mle-model)\n    3. [Training RL Model](#training-rl-model)\n    4. [Training Evaluator](#training-evaluator)\n    5. [Training GGNN](#training-ggnn)\n\n# Results\nThese are the reproduction results from this repository as compared to the paper\n\n| Training Type | Num first points | LSTM Beam Size | Before | Now   |\n|:-------------:|:----------------:|:--------------:|:------:|:-----:|\n| MLE + Att     | 1                | 1              | 65.43  | 66.35 |\n| MLE + Att + RL | 1               | 1              | 67.17  | 67.45 |\n| MLE + Att + Evaluator | 5        | 1              | 69.72  | 71.05 |\n| MLE + Att + Evaluator | 5        | 8              | 70.21  | 70.91 |\n| MLE + Att + Evaluator + GGNN | 5 | 8              | 71.38  | 72.05 |\n| MLE + Att + Evaluator + GGNN | 5 | 1              |   -    | 72.08 |\n| MLE + Att + Evaluator + GGNN (Shared Encoder) | 5 | 8 | -  | 72.22 |\n| MLE + Att + Evaluator + GGNN (Shared Encoder) | 5 | 1 | -  | **72.33** |\n\n**Note:** Benchmarked forward pass speed for the tool (with 5 first points and 1 beam size) is 0.3 seconds per interaction on a TitanXp\n\n**Note:** Shared Encoder refers to sharing the Resnet between the graph network and the convLSTM network. In the original paper, the two networks were kept separate.\n\n# Environment Setup\nAll the code has been run and tested on Ubuntu 16.04, Python 2.7.12, Pytorch 0.4.0, CUDA 9.0, TITAN X/Xp and GTX 1080Ti GPUs\n\n- Get code after [signing up](http://www.cs.toronto.edu/polyrnn/code_signup/)\n- Go into the downloaded code directory\n```\ncd \u003cpath_to_downloaded_directory\u003e\n```\n- Setup python environment\n```\nvirtualenv env\nsource env/bin/activate\npip install -r requirements.txt\n```\n- Add the project to PYTHONPATH\n```\nexport PYTHONPATH=$PWD\n```\n\n# Tool\n- [Setup](#environment-setup) your environment\n- Download the MLE+RL+Evaluator+GGNN model after getting access by [signing up](http://www.cs.toronto.edu/polyrnn/code_signup/)\n\n## Backend\n- Launch backend (flask server) with,\n```\npython Tool/tool.py --exp Experiments/tool.json --reload \u003cpath_to_model\u003e --port \u003cport\u003e --image_dir Tool/frontend/static/img/\n```\n\n## Frontend\n- Edit Tool/frontend/static/js/polygon.js and change globalFolder to the appropriate\ndirectory based on where you cloned the repository.\n- With python2.7, run\n```\ncd Tool/frontend/\npython -m SimpleHTTPServer\n```\n- On your browser, navigate to localhost:8000. You should see a page like\n\u003cimg src = \"Docs/tool.png\" width=\"100%\"/\u003e\n\n**Note:** Replace SimpleHTTPServer with http.server if you are using python3 for the server\n\n**Note:** You can setup your own image directory by editing Tool/frontend/static/js/polygon.js and passing that path to Tool/tool.py\nfrom the command line. This image directory MUST contain the pre-defined images that are defined in Tool/frontend/index.html\n\n# Testing Models\n- [Setup](#environment-setup) your environment\n- Download pretrained models after getting access by [signing up](http://www.cs.toronto.edu/polyrnn/code_signup/)\n\n```\npython Scripts/prediction/generate_annotation.py --exp \u003cpath_to_corresponding_experiment\u003e --reload \u003cpath_to_checkpoint\u003e --output_dir \u003cpath_to_store_predictions\u003e\n```\n- If you are testing a RL checkpoint, then the experiment file would correspond be the RL experiment file, and similarly for other stages of the model\n- You can check predicted/GT masks for every instance in the output_dir\n- To get scores, run\n```\npython Scripts/get_scores.py --pred \u003cpath_to_preds\u003e --output \u003cpath_to_file_to_save_results\u003e\n```\n\n# Training Models\n\n## Data \n\n### Cityscapes\n- Download the Cityscapes dataset (leftImg8bit\\_trainvaltest.zip) from the official [website](https://www.cityscapes-dataset.com/downloads/) [11 GB]\n- Our processed annotation files are included in the download file you get after signing up\n- From the root directory, run the following command with appropriate paths to get the annotation files ready for your machine\n```\npython Scripts/data/change_paths.py --city_dir \u003cpath_to_downloaded_leftImg8bit_folder\u003e --json_dir \u003cpath_to_downloaded_annotation_file\u003e --out_dir \u003coutput_dir\u003e\n```\n\n### Custom Dataset\nTo train on your custom datasets, you have one of two options:\n- Prepare annotation data similar to our annotation files and use our default DataProvider\n- Implement your own DataProvider following the cityscapes implementation for your own data\n\n## Training\n- [Setup](#environment-setup) your environment\n- Download the pre-trained Pytorch Resnet-50 from [here](https://download.pytorch.org/models/resnet50-19c8e357.pth)\n- **Note** - While resuming training, always resume from end of epoch checkpoints to produce reproducible results!\n\n### Training MLE model\n- Edit the experiment file at Experiments/mle.json and change paths for your machine\n- From the root directory, run\n```\npython Scripts/train/train_ce.py --exp Experiments/mle.json --resume \u003coptional_if_resuming_training\u003e\n```\n- You can view progress on Tensorboard (logs are at \u003cexperiment\\_dir\u003e/logs/)\n\n### Training RL model\n- Edit the experiment file at Experiments/rl.json and change paths for your machine\n- In the experiment file, set xe\\_initializer to the best MLE model\n- From the root directory, run\n```\npython Scripts/train/train_rl.py --exp Experiments/rl.json --resume \u003coptional_if_resuming_training\u003e\n```\n- **Note** - You might have to play with hyperparameters a bit to achieve stable training, especially temperature, lr and lr\\_decay\n\n### Training Evaluator\n- Edit the experiment file at Experiments/evaluator.json and change paths for your machine\n- In the experiment file, set xe\\_initializer to the best RL model\n- From the root directory, run\n```\npython Scripts/train/train_evaluator.py --exp Experiments/evaluator.json --resume \u003coptional_if_resuming_training\u003e\n```\n\n### Training GGNN\n- Edit the experiment file at Experiments/ggnn.json and change paths for your machine\n- In the experiment file, set xe\\_initializer to the best Evaluator model\n- From the root directory, run\n```\npython Scripts/train/train_ggnn.py --exp Experiments/ggnn.json --resume \u003coptional_if_resuming_training\u003e\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffidler-lab%2Fpolyrnn-pp-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffidler-lab%2Fpolyrnn-pp-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffidler-lab%2Fpolyrnn-pp-pytorch/lists"}