{"id":13785463,"url":"https://github.com/solcummings/earthvision2021-weakly-supervised","last_synced_at":"2025-05-11T21:30:37.173Z","repository":{"id":53059544,"uuid":"378041057","full_name":"solcummings/earthvision2021-weakly-supervised","owner":"solcummings","description":"1st Place Solution for the EARTHVISION 2021 DynamicEarthNet Challenge - Weakly-Supervised Multi-Class Change Detection Track at CVPRW 2021 ","archived":false,"fork":false,"pushed_at":"2022-11-03T23:33:45.000Z","size":6375,"stargazers_count":20,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-17T21:39:22.502Z","etag":null,"topics":["change-detection","deep-learning","pytorch","remote-sensing","semantic-segmentation","siamese-neural-network"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/solcummings.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-06-18T05:12:54.000Z","updated_at":"2024-08-12T20:14:01.000Z","dependencies_parsed_at":"2023-01-21T14:00:17.456Z","dependency_job_id":null,"html_url":"https://github.com/solcummings/earthvision2021-weakly-supervised","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solcummings%2Fearthvision2021-weakly-supervised","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solcummings%2Fearthvision2021-weakly-supervised/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solcummings%2Fearthvision2021-weakly-supervised/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solcummings%2Fearthvision2021-weakly-supervised/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/solcummings","download_url":"https://codeload.github.com/solcummings/earthvision2021-weakly-supervised/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253638728,"owners_count":21940414,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["change-detection","deep-learning","pytorch","remote-sensing","semantic-segmentation","siamese-neural-network"],"created_at":"2024-08-03T19:01:00.726Z","updated_at":"2025-05-11T21:30:32.733Z","avatar_url":"https://github.com/solcummings.png","language":"Python","funding_links":[],"categories":["Hyperspectral"],"sub_categories":["Deep Learning"],"readme":"# Siamese Attention U-Net for Multi-Class Change Detection\n1st place solution for the EARTHVISION 2021 DynamicEarthNet Challenge - Weakly-Supervised Multi-Class Change Detection Track at CVPRW 2021  \n[[Challenge Site]](https://competitions.codalab.org/competitions/30441)\n[[Presentation Slides]](./examples/earthvision2021_presentation.pdf)\n[[Presentation Video]](https://youtu.be/nrntHeY8vtg?t=11994)\n[[Dataset]](https://competitions.codalab.org/competitions/30441#participate)\n\n## Overview\nThis work introduces a pixel-wise change detection network named Siamese Attention U-Net that incorporates attention mechanisms in the Siamese U-Net architecture. Experiments show the architectural change alongside training strategies such as semi-supervised learning produce more robust models.  \n![Siamese Attention U-Net](./examples/siamese_attention_unet.png)\n\n### Results\n1. Attention block  \n![Proposed Attention Block](./examples/attention_block_proposed.png)\n\nWhen training the model with Jaccard loss, attending the up-sampled features improves mean IoU scores whereas attending skipped features - originally proposed in Attention UNet - do not. The improvement implies a need for prioritizing information in the coarser resolution features.\n\n|Attended features|mean IoU (val)|\n|:-:|:-:|\n|None|0.2635|\n|Skipped features|0.2603|\n|Up-sampled features|0.2658|\n\n2. Loss function  \n\nJaccard loss and Dice loss optimize for different metrics, producing slightly varying results. Ensembling models trained exclusively on each loss function improves scores.\n\n|Loss function|mean IoU (val)|\n|:-:|:-:|\n|Jaccard|0.2658|\n|Dice|0.2668|\n|Ensemble|0.2676|\n\n3. Semi-supervised learning  \n\nCreating hard pseudo labels for the public validation and test dataset, then retraining the model alongside the original training dataset improves scores regardless of loss function. The ensemble of models trained on each loss function is submitted to the public test benchmark.\n\n|Pseudo labels|Loss function|mean IoU (val)|mean IoU (test)|\n|:-:|:-:|:-:|:-:|\n|None|Jaccard|0.2658||\n|val+test|Jaccard|0.2669||\n|None|Dice|0.2668|\n|val+test|Dice|0.2674||\n|None|Ensemble|0.2676||\n|val+test|Ensemble|0.2684|0.2423|\n\n\n## Usage\n### Dependencies\n- gdal\n- numpy\n- pandas\n- pillow\n- pytorch\n- pyyaml\n- torchvision\n- tqdm\n\n### Downloading and Preprocessing Data\n```bash\n# assumes all data is unzipped to ./earthvision2021/data/source/\ncd ./setup/\n# convert data and organize directories\nbash setup.bash\n```\n```bash\ncd ./earthvision2021/preprocessing/\n# create csv file that determines crop regions for training\npython crop.py\nmv train.csv ../data/train/random_128_binary_attempts128.csv\n```\n\n### Training\n```bash\ncd ./earthvision2021/\n# train model according to configurations in ./earthvision2021/config_train.yml\npython train.py\n```\n\n### Predicting\n```bash\ncd ./earthvision2021/\n# predict using model according to configurations in ./earthvision2021/config_predict.yml\npython predict.py\n```\n\n### Pseudo Labelling \n```bash\ncd ./earthvision2021/postprocessing/\n# binarizing prediction results to create hard pseudo labels\npython label_binarize.py --in_dir ${prediction_dir}\ncd ../preprocessing/\n# create new csv file of crop regions\npython crop.py --subset_file ${filename_text_file}\nmv train.csv ../data/train/${csv_file}\n# change \"aoi_file\" in ../config_train.yml to ${csv_file} and retrain\n```\n\n### Postprocessing\n```bash\ncd ./earthvision2021/postprocessing/\n# ensemble multiple predictions results\npython ensemble.py --in_dir_list ${prediction_dir} ...\n# binarize prediction results\npython binarize.py --in_dir ${prediction_dir}\n```\n\n## Citation\nIf you find this work useful in your research or publication, please cite this work:\n```TeX\n@inproceedings{cummings2022_sia,\n  author={Cummings, Sol and Kondmann, Lukas and Zhu, Xiao Xiang},\n  booktitle={IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium},\n  title={Siamese Attention U-Net for Multi-Class Change Detection},\n  year={2022},\n  pages={211-214},\n  doi={10.1109/IGARSS46834.2022.9884834}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolcummings%2Fearthvision2021-weakly-supervised","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsolcummings%2Fearthvision2021-weakly-supervised","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolcummings%2Fearthvision2021-weakly-supervised/lists"}