{"id":13734834,"url":"https://github.com/dlr-eoc/ukis-csmask","last_synced_at":"2025-04-10T18:54:12.351Z","repository":{"id":43400977,"uuid":"328616234","full_name":"dlr-eoc/ukis-csmask","owner":"dlr-eoc","description":"UKIS Cloud Shadow MASK","archived":false,"fork":false,"pushed_at":"2023-12-05T13:42:53.000Z","size":72928,"stargazers_count":45,"open_issues_count":0,"forks_count":6,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-05-16T02:33:47.974Z","etag":null,"topics":["cloud-shadow","cloud-shadow-mask","earth-observation","gis","satellite-data","ukis"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dlr-eoc.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.rst","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-01-11T09:45:08.000Z","updated_at":"2024-01-04T18:34:27.000Z","dependencies_parsed_at":"2023-12-15T21:17:26.307Z","dependency_job_id":"f6cee9b4-1344-4036-8aa7-41e0d658f0ba","html_url":"https://github.com/dlr-eoc/ukis-csmask","commit_stats":{"total_commits":63,"total_committers":6,"mean_commits":10.5,"dds":0.5079365079365079,"last_synced_commit":"91938f279b2e695e08fd557c190128b5ab0a2cfa"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlr-eoc%2Fukis-csmask","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlr-eoc%2Fukis-csmask/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlr-eoc%2Fukis-csmask/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlr-eoc%2Fukis-csmask/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dlr-eoc","download_url":"https://codeload.github.com/dlr-eoc/ukis-csmask/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248275365,"owners_count":21076581,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-shadow","cloud-shadow-mask","earth-observation","gis","satellite-data","ukis"],"created_at":"2024-08-03T03:01:00.379Z","updated_at":"2025-04-10T18:54:07.337Z","avatar_url":"https://github.com/dlr-eoc.png","language":"Python","funding_links":[],"categories":["`Python` processing of optical imagery (non deep learning)"],"sub_categories":["Processing imagery - post processing"],"readme":"# [![UKIS](img/ukis-logo.png)](https://www.dlr.de/eoc/en/desktopdefault.aspx/tabid-5413/10560_read-21914/) ukis-csmask\n\n![ukis-csmask](https://github.com/dlr-eoc/ukis-csmask/workflows/ukis-csmask/badge.svg)\n[![codecov](https://codecov.io/gh/dlr-eoc/ukis-csmask/branch/main/graph/badge.svg)](https://codecov.io/gh/dlr-eoc/ukis-csmask)\n![Upload Python Package](https://github.com/dlr-eoc/ukis-csmask/workflows/Upload%20Python%20Package/badge.svg)\n[![PyPI version](https://img.shields.io/pypi/v/ukis-csmask)](https://pypi.python.org/pypi/ukis-csmask/)\n[![GitHub license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)\n[![Code Style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://black.readthedocs.io/en/stable/)\n[![DOI](https://zenodo.org/badge/328616234.svg)](https://zenodo.org/badge/latestdoi/328616234)\n\nUKIS Cloud Shadow MASK (ukis-csmask) package masks clouds and cloud shadows in Sentinel-2, Landsat-9, Landsat-8, Landsat-7 and Landsat-5 images. Masking is performed with a pre-trained convolution neural network. It is fast and works directly on Level-1C data (no atmospheric correction required). Images just need to be in Top Of Atmosphere (TOA) reflectance and include at least the \"blue\", \"green\", \"red\" and \"nir\" spectral bands. Best performance (in terms of accuracy and speed) is achieved when images also include \"swir16\" and \"swir22\" spectral bands and are resampled to approximately 30 m spatial resolution.\n\nThis [publication](https://doi.org/10.1016/j.rse.2019.05.022) provides further insight into the underlying algorithm and compares it to the widely used [Fmask](http://www.pythonfmask.org/en/latest/) algorithm across a heterogeneous test dataset.\n\n\u003e Wieland, M.; Li, Y.; Martinis, S. Multi-sensor cloud and cloud shadow segmentation with a convolutional\nneural network. *Remote Sensing of Environment*, 2019, 230, 1-12. [https://doi.org/10.1016/j.rse.2019.05.022](https://doi.org/10.1016/j.rse.2019.05.022)\n\nThis [publication](https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-217-2022) introduces the Python package, performs additional evaluation on recent cloud and cloud shadow benchmark datasets and tests the applicability of ukis-csmask on Landsat-9 imagery.\n\n\u003e Wieland, M.; Fichtner, F.; Martinis, S. UKIS-CSMASK: A Python package for multi-sensor cloud and cloud shadow segmentation. *Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.*, 2022, XLIII-B3-2022, 217–222. [https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-217-2022](https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-217-2022)\n\nIf you use ukis-csmask in your work, please consider citing one of the above publications.\n\n![Examples](img/examples.png)\n\n## Example (Sentinel 2)\nHere's an example on how to compute a cloud and cloud shadow mask from an image. Please note that here we use [ukis-pysat](https://github.com/dlr-eoc/ukis-pysat) for convencience image handling, but you can also work directly with [numpy](https://numpy.org/) arrays.\n\n````python\nfrom ukis_csmask.mask import CSmask\nfrom ukis_pysat.raster import Image, Platform\n\n# read Level-1C image from file, convert digital numbers to TOA reflectance\n# and make sure resolution is 30 m to get best performance\nimg = Image(data=\"sentinel2.tif\", dimorder=\"last\")\nimg.dn2toa(platform=Platform.Sentinel2)\nimg.warp(\n    resampling_method=0,\n    resolution=30,\n    dst_crs=img.dataset.crs\n)\n\n# compute cloud and cloud shadow mask\n# NOTE: band_order must match the order of bands in the input image. it does not have to be in this explicit order.\n# make sure to use these six spectral bands to get best performance\ncsmask = CSmask(\n    img=img.arr,\n    band_order=[\"blue\", \"green\", \"red\", \"nir\", \"swir16\", \"swir22\"],\n    nodata_value=0,\n)\n\n# access cloud and cloud shadow mask\ncsmask_csm = csmask.csm\n\n# access valid mask\ncsmask_valid = csmask.valid\n\n# convert results to UKIS-pysat Image\ncsmask_csm = Image(csmask.csm, transform=img.dataset.transform, crs=img.dataset.crs, dimorder=\"last\")\ncsmask_valid = Image(csmask.valid, transform=img.dataset.transform, crs=img.dataset.crs, dimorder=\"last\")\n\n# write results back to file\ncsmask_csm.write_to_file(\"sentinel2_csm.tif\", dtype=\"uint8\", compress=\"PACKBITS\")\ncsmask_valid.write_to_file(\"sentinel2_valid.tif\", dtype=\"uint8\", compress=\"PACKBITS\", kwargs={\"nbits\":2})\n````\n\n## Example (Landsat 8)\nHere's a similar example based on Landsat 8.\n\n````python\nimport rasterio\nimport numpy as np\nfrom ukis_csmask.mask import CSmask\nfrom ukis_pysat.raster import Image, Platform\n\n# set Landsat 8 source path and prefix (example)\ndata_path = \"/your_data_path/\"\nL8_file_prefix = \"LC08_L1TP_191015_20210428_20210507_02_T1\"\n\ndata_path = data_path+L8_file_prefix+\"/\"\nmtl_file  = data_path+L8_file_prefix+\"_MTL.txt\"\n\n# stack [B2:'Blue', B3:'Green', B4:'Red', B5:'NIR', B6:'SWIR1', B7:'SWIR2'] as numpy array\nL8_band_files  = [data_path+L8_file_prefix+'_B'+ x + '.TIF' for x in [str(x+2) for x in range(6)]]\n\n# \u003e\u003e adopted from https://gis.stackexchange.com/questions/223910/using-rasterio-or-gdal-to-stack-multiple-bands-without-using-subprocess-commands\n# read metadata of first file\nwith rasterio.open(L8_band_files[0]) as src0:\n    meta = src0.meta\n# update meta to reflect the number of layers\nmeta.update(count = len(L8_band_files))\n# read each layer and append it to numpy array\nL8_bands = []\nfor id, layer in enumerate(L8_band_files, start=1):\n    with rasterio.open(layer) as src1:\n        L8_bands.append(src1.read(1))\nL8_bands = np.stack(L8_bands,axis=2)\n# \u003c\u003c\n\nimg = Image(data=L8_bands, crs = meta['crs'], transform = meta['transform'], dimorder=\"last\")\n\nimg.dn2toa(\n        platform=Platform.Landsat8,\n        mtl_file=mtl_file,\n        wavelengths = [\"blue\", \"green\", \"red\", \"nir\", \"swir16\", \"swir22\"]\n)\n# \u003e\u003e proceed by analogy with Sentinel 2 example\n````\n\n## Installation\nThe easiest way to install ukis-csmask is through pip. To install ukis-csmask with [default CPU provider](https://onnxruntime.ai/docs/execution-providers/) run the following.\n\n```shell\npip install ukis-csmask[cpu]\n```\n\nTo install ukis-csmask with [OpenVino support](https://onnxruntime.ai/docs/execution-providers/OpenVINO-ExecutionProvider.html) for enhanced CPU inference run the following instead.\n\n```shell\npip install ukis-csmask[openvino]\n```\n\nTo install ukis-csmask with [GPU support](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html) run the following instead. This requires that you have a GPU with CUDA runtime libraries installed on the system.\n\n```shell\npip install ukis-csmask[gpu]\n```\n\nukis-csmask depends on [onnxruntime](https://onnxruntime.ai/). For a list of additional dependencies check the [requirements](https://github.com/dlr-eoc/ukis-csmask/blob/main/requirements.txt).\n\n## Contributors\nThe UKIS team creates and adapts libraries which simplify the usage of satellite data. Our team includes (in alphabetical order):\n* Boehnke, Christian\n* Fichtner, Florian\n* Mandery, Nico\n* Martinis, Sandro\n* Riedlinger, Torsten\n* Wieland, Marc\n\nGerman Aerospace Center (DLR)\n\n## Licenses\nThis software is licensed under the [Apache 2.0 License](https://github.com/dlr-eoc/ukis-csmask/blob/main/LICENSE).\n\nCopyright (c) 2020 German Aerospace Center (DLR) * German Remote Sensing Data Center * Department: Geo-Risks and Civil Security\n\n## Changelog\nSee [changelog](https://github.com/dlr-eoc/ukis-csmask/blob/main/CHANGELOG.rst).\n\n## Contributing\nThe UKIS team welcomes contributions from the community.\nFor more detailed information, see our guide on [contributing](https://github.com/dlr-eoc/ukis-csmask/blob/main/CONTRIBUTING.md) if you're interested in getting involved.\n\n## What is UKIS?\nThe DLR project Environmental and Crisis Information System (the German abbreviation is UKIS, standing for [Umwelt- und Kriseninformationssysteme](https://www.dlr.de/eoc/en/desktopdefault.aspx/tabid-5413/10560_read-21914/) aims at harmonizing the development of information systems at the German Remote Sensing Data Center (DFD) and setting up a framework of modularized and generalized software components.\n\nUKIS is intended to ease and standardize the process of setting up specific information systems and thus bridging the gap from EO product generation and information fusion to the delivery of products and information to end users.\n\nFurthermore the intention is to save and broaden know-how that was and is invested and earned in the development of information systems and components in several ongoing and future DFD projects.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdlr-eoc%2Fukis-csmask","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdlr-eoc%2Fukis-csmask","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdlr-eoc%2Fukis-csmask/lists"}