{"id":20469022,"url":"https://github.com/pfnet-research/superpixel-align","last_synced_at":"2025-08-02T07:15:06.238Z","repository":{"id":86583221,"uuid":"163373737","full_name":"pfnet-research/superpixel-align","owner":"pfnet-research","description":"Official implementation of \"Minimizing Supervision for Free-space Segmentation\" paper","archived":false,"fork":false,"pushed_at":"2018-12-28T06:19:33.000Z","size":6472,"stargazers_count":24,"open_issues_count":2,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-29T11:38:15.410Z","etag":null,"topics":["deep-learning","semantic-segmentation","superpixels"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1711.05998","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pfnet-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-12-28T06:19:08.000Z","updated_at":"2025-02-24T23:36:09.000Z","dependencies_parsed_at":null,"dependency_job_id":"39c6e8c9-e2fa-4631-b981-55c6f9c05e06","html_url":"https://github.com/pfnet-research/superpixel-align","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pfnet-research/superpixel-align","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfnet-research%2Fsuperpixel-align","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfnet-research%2Fsuperpixel-align/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfnet-research%2Fsuperpixel-align/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfnet-research%2Fsuperpixel-align/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pfnet-research","download_url":"https://codeload.github.com/pfnet-research/superpixel-align/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfnet-research%2Fsuperpixel-align/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268348447,"owners_count":24236297,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-02T02:00:12.353Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","semantic-segmentation","superpixels"],"created_at":"2024-11-15T14:07:39.185Z","updated_at":"2025-08-02T07:15:06.176Z","avatar_url":"https://github.com/pfnet-research.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"Superpixel Align\n================\n\nThis is the official implementation of [\"Minimizing Supervision for Free-space Segmentation\n\"](https://arxiv.org/abs/1711.05998). BibTeX is [here](#reference).\n\nTested environment\n------------------\n\n- Ubuntu 16.04\n- CUDA 9.0\n\nSetup environment\n-----------------\n\n```bash\nbash setup.sh\n```\n\nThe above shell script creates miniconda environment under this directory and install requirements below. The `setup.sh` installs all of them **so you don't need to install the dependencies below by yourself.**\n\n- Python: \u003e=3.6.2\n- External libraries: openmpi(--with-cuda)==2.1.1, cuDNN==7.0.3, nccl2==2.1.2\n- Python packages: cython==0.27.3, tqdm==4.19.4, Pillow==4.3.0, scipy==1.0.0, scikit-image==0.13.1, opencv-python==3.3.0.10, pytorch==0.2.0, mpi4py==3.0.0, cupy==4.0.0b1, chainer==4.0.0b1, chainermn==git+https://github.com/mitmul/chainermn@support-cupy-v4.0.0b1, chainercv==0.7.0\n\nPrepare\n-------\n\nBefore starting all procedures below, please run this first to activate the environment:\n\n```bash\nsource miniconda/bin/activate road-segm\n```\n\n### 1. Download Cityscapes dataset\n\nPlease make an account at the [Cityscapes dataset](https://www.cityscapes-dataset.com) site and download the files below from [the donwload page](https://www.cityscapes-dataset.com/downloads/). You need these files:\n\n- [gtFine_trainvaltest.zip (241MB)](https://www.cityscapes-dataset.com/file-handling/?packageID=1)\n- [gtCoarse.zip (1.3GB)](https://www.cityscapes-dataset.com/file-handling/?packageID=2)\n- [leftImg8bit_trainvaltest.zip (11GB)](https://www.cityscapes-dataset.com/file-handling/?packageID=3)\n- [leftImg8bit_trainextra.zip (44GB)](https://www.cityscapes-dataset.com/file-handling/?packageID=4)\n\nAfter you downloaded them, please extract all files in a same directory. That directory is referred as `[CITYSCAPES_DIR]` in the descriptions below.\n\n### 2. Create symlink to the cityscapes dir\n\n```bash\nln -s [CITYSCAPES_DIR] data/cityscapes\n```\n\n[CITYSCAPES_DIR] should be replaced with the path to the directory of the Cityscapes dataset. The directory should contains `gtCoarse`, `gtFine`, and `leftImg8bit` dirs. Each subdir should have the dirs below:\n\n- data/cityscapes\n  - gtCoarse\n    - test\n    - train\n    - train_extra\n    - val\n  - gtFine\n    - test\n    - train\n    - val\n  - leftImg8bit\n    - test\n    - train\n    - train_extra\n    - val\n\n### 3. Convert PyTorch model to Chainer model:\n\n```bash\ncd models\npython convert_pth2ch.py\ncd ..\n```\n\n### 4. Create zip files (dataset creation)\n\n```bash\nbash utils/create_zip_files.sh\n```\n\nClustering superpixel align features\n------------------------------------\n\nThroughout all the commands below,\n\n- Please replace `[NUMBER OF GPUS]` with the number of GPUs you want to use for this script.\n- The GPU ID starts counting from 0, so if you want to specify the GPU IDs, please set `CUDA_VISIBLE_DEVICES` environment variable.\n- Please run this first to activate the environment:\n\n  ```bash\n  source miniconda/bin/activate road-segm\n  ```\n\n### Generate labels of randomly selected 300 train images and evaluate them\n\n```bash\nMPLBACKEND=Agg bash utils/create_random300_labels.sh [NUMBER OF GPUS]\n\n# Wait until it finishes (Check the processes has terminated)\n\npython utils/mean_result.py \\\nresults/estimated_train_random300_labels/result.json \\\n--n_imgs 300 --count_duplicated\n```\n\nThe evaluation result is found [here](#random_300).\n\n### Generate labels of validation images and evaluate them\n\n```bash\nMPLBACKEND=Agg bash utils/create_val_labels.sh [NUMBER OF GPUS]\n\n# Wait until it finishes (Check the processes has terminated)\n\n# Evaluate the estimation\npython utils/mean_result.py results/estimated_val_labels/result.json\n```\n\nThe evaluation result is found [here](#val).\n\n### Generate labels of training images and evaluate them\n\n```bash\nMPLBACKEND=Agg bash utils/create_train_labels.sh [NUMBER OF GPUS]\n\n# Wait until it finishes (Check the processes has terminated)\n\n# Evaluate the estimation\npython utils/mean_result.py results/estimated_train_labels/result.json\n\n# Zip the generated labels\nfind results/estimated_train_labels -name \"*leftImg8bit.npy\" | zip -0r results/estimated_train_labels.0.zip -@\n```\n\nThen, please make sure that all the files below exist under `results` dir.\n\n- estimated_train_labels.0.zip\n\nThis file is used as labels to train SegNet.\n\nTrain SegNet on estimated labels\n--------------------------------\n\n- Please change the value given to `--n_gpus` option to the number of GPUs you want to use for this experiment.\n- Please change the value given to `--batchsize` option to the number you preferred when the default value 8 is too big to fit in your GPUs.\n- The GPU ID starts counting from 0, so if you want to specify the GPU IDs, please set `CUDA_VISIBLE_DEVICES` environment variable.\n- Please run this first to activate the environment:\n\n  ```bash\n  source miniconda/bin/activate road-segm\n  ```\n\n\n### Train SegNet on generated labels\n\nThe command below starts training of SegNetBasic on the generated labels created by the above script. The training goes for 2000 iterations with Adam optimizer.\n\n```bash\nLD_LIBRARY_PATH=miniconda/envs/road-segm/lib:$LD_LIBRARY_PATH \\\nMPLBACKEND=Agg \\\npython utils/run_train_rounds.py \\\n--img_zip_fn data/cityscapes_train_imgs.0.zip \\\n--label_zip_fn data/cityscapes_train_labels.0.zip \\\n--estimated_label_zip results/estimated_train_labels.0.zip \\\n--n_gpus 8 \\\n--batchsize 8\n```\n\n**See the `log` file in the `results/train_round1_[DATETIME]` dir for the evaluation scores of the generated labels by the trained SegNet on the estimated labels by superpixel align feature clustering.**\n\n\nEvaluation results\n------------------\n\n### Superpixel align feature clustering\n\n#### Randomly selected 300 train images\u003ca name=\"random_300\"\u003e\u003c/a\u003e\n\n| Metric    | Value              |\n|:----------|:-------------------|\n| Road IoU  | 0.8129520227337709 |\n| Precision | 0.8835840497695169 |\n| Recall    | 0.9166856000528959 |\n\n#### Validation images\u003ca name=\"val\"\u003e\u003c/a\u003e\n\n| Metric    | Value              |\n|:----------|:-------------------|\n| Road IoU  | 0.7619056844993818 |\n| Precision | 0.8799825987212356 |\n| Recall    | 0.8919905105061199 |\n\n### After SegNet training\u003ca name=\"segnet\"\u003e\u003c/a\u003e\n\n#### Evaluated on validation images\n\n| Metric    | Value              |\n|:----------|:-------------------|\n| Road IoU  | 0.8345039286452565 |\n| Precision | 0.897570349944977  |\n| Recall    | 0.9232502418464443 |\n\nReference\u003ca name='reference'\u003e\u003c/a\u003e\n---------------------------------\n\n```\n@InProceedings{Tsutsui_2018_CVPR_Workshops,\n  author = {Tsutsui, Satoshi and Kerola, Tommi and Saito, Shunta and Crandall, David J.},\n  title = {Minimizing Supervision for Free-Space Segmentation},\n  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},\n  month = {June},\n  year = {2018}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpfnet-research%2Fsuperpixel-align","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpfnet-research%2Fsuperpixel-align","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpfnet-research%2Fsuperpixel-align/lists"}