{"id":21581421,"url":"https://github.com/dlr-eoc/ukis_sat2h5","last_synced_at":"2026-04-12T16:03:23.216Z","repository":{"id":209626497,"uuid":"724523536","full_name":"dlr-eoc/UKIS_sat2h5","owner":"dlr-eoc","description":"Convert geospatial satellite images to H5 arrays and back keeping the spatial extent.","archived":false,"fork":false,"pushed_at":"2023-11-28T09:11:53.000Z","size":530,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-24T14:44:31.876Z","etag":null,"topics":["array","dask","deep-learning","geospatial","hdf5","numpy","python","python3","raster","satellite-imagery","tiff"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dlr-eoc.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-11-28T08:56:10.000Z","updated_at":"2024-04-16T06:07:53.000Z","dependencies_parsed_at":"2023-11-28T11:29:42.062Z","dependency_job_id":"940a1add-8f0c-4bd3-8c2b-cf1013bb515d","html_url":"https://github.com/dlr-eoc/UKIS_sat2h5","commit_stats":null,"previous_names":["dlr-eoc/ukis_sat2h5"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlr-eoc%2FUKIS_sat2h5","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlr-eoc%2FUKIS_sat2h5/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlr-eoc%2FUKIS_sat2h5/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlr-eoc%2FUKIS_sat2h5/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dlr-eoc","download_url":"https://codeload.github.com/dlr-eoc/UKIS_sat2h5/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244181390,"owners_count":20411605,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["array","dask","deep-learning","geospatial","hdf5","numpy","python","python3","raster","satellite-imagery","tiff"],"created_at":"2024-11-24T14:12:27.982Z","updated_at":"2025-10-17T22:25:13.977Z","avatar_url":"https://github.com/dlr-eoc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![UKIS](docs/ukis-logo.png)](https://www.dlr.de/eoc/en/desktopdefault.aspx/tabid-5413/10560_read-21914/) UKIS_sat2h5\n=======================================\n\n`ukis_sat2h5` can be used to convert many (similar sized) GeoTiff images into a `HDF-5` array while\nkeeping track of spatial extent and projections. `HDF-5` arrays are easier to use in deep learning\napplications. All images will be padded to fit the same dimensions. `ukis_sat2h5` also includes\ntools for tiling images (with optional overlap) and converting images back to GeoTiff files.\n\n![Workflow overview](docs/overview.svg)\n\nImagine a folder structure like this:\n\n```\nimages\n├── image1\n│   ├── image.tif\n│   └── label.tif\n├── image2\n│   ├── image.tif\n│   └── label.tif\n└── image3\n    ├── image.tif\n    └── label.tif\n```\n\n`ukis_sat2h5` will convert this into a `HDF-5` file with the following structure:\n\n```\nimages.hdf5\n  /affine    : Dataset {\u003cnimages\u003e, 6}\n  /epsg      : Dataset {\u003cnimages\u003e}\n  /img       : Dataset {\u003cnimages\u003e, \u003cnbands\u003e, \u003cnrows\u003e, \u003cncols\u003e}\n  /img_means : Dataset {\u003cnbands\u003e}\n  /img_stds  : Dataset {\u003cnbands\u003e}\n  /lbl       : Dataset {\u003cnimages\u003e, \u003cnrows\u003e, \u003cncols\u003e}\n  /path      : Dataset {\u003cnimages\u003e}\n```\n\n## Examples\n\n```shell\n$ python -m ukis_sat2h5 img_to_h5 -r ~/tmp/sen2_ref_images -d ~/tmp/sen2_ref_images.h5 -f \"*.vrt\" -l \"*label*\" -b 2 3 4 8\n$ python -m ukis_sat2h5 tile_h5 -s ~/tmp/sen2_ref_images.h5 -d ~/tmp/sen2_ref_images_tiled.h5 -t 256 -o 128 -a 1024\n$ python -m ukis_sat2h5 h5_to_image -s ~/tmp/sen2_ref_images_tiled.h5 -d ~/tmp/tiles_to_images -i 1 42 128\n```\n\n\n# Installation\n\nThis project uses a `conda` environment. For installing dependencies use:\n\n```bash\nconda env create\n```\n\n## Dependencies\nFor the latest list of dependencies check the [`environment.yml`](environment.yml).\n\n\n# Usage\n\n## CLI\n\nThis package provides standalone CLI functionality\n\n```\nusage: ukis_sat2h5 [-h] {img_to_h5,h5_to_img,tile_h5} ...\n\nConvert geographic images to H5 file and back and tile them for Deep Learning applications\n\npositional arguments:\n  {img_to_h5,h5_to_img,tile_h5}\n                        Tools\n    img_to_h5           Convert geographic images to H5 arrays\n    h5_to_img           Convert H5 arrays (created with this tool) to back to geographic images\n    tile_h5             Tile H5 array and preserve geographic location\n\noptions:\n  -h, --help            show this help message and exit\n```\n\n## Python Module\n\nThe main functionality is encapsulated inside the follwing functions\n\n```\nimport ukis_sat2h5\n\nukis_sat2h5.conversion.convert_img_to_h5(src_path, dst_file, file_glob, image_bands)\nukis_sat2h5.conversion.convert_h5_to_img(src_file, dst_folder, index)\nukis_sat2h5.tiling.tile_h5(h5_file, h5_file_tiled, tile_size, overlap, target_size)\n```\n\n\n# Development\n\n## `pre-commit`\n\nSome development guardrails are enforced via [`pre-commit`](https://pre-commit.com/). This is to\nensure we follow similar code styles or it automatically cleans up jupyter notebooks.\n\nTo install `pre-commit` (not necessary if you [installed the conda\nenvironment](#install-conda-evnironment)):\n\n```shell\n$ # conda/pip install pre-commit\n```\n\nTo initialize all pre-commit hooks, run:\n\n```shell\n$ pre-commit install\n```\n\nTo test whether `pre-commit` works:\n\n```shell\n$ pre-commit run --all-files\n```\n\nIt will check all files tracked by git and apply the triggers set up in\n[`.pre-commit-config.yaml`](.pre-commit-config.yaml). That is, it will run triggers, possibly\nchanging the contents of the file (e.g. `black` formatting). Once set up, `pre-commit` will run, as\nthe name implies, prior to each `git commit`. In its current config, it will format code with\n`black` and `isort`, clean up `jupyter notebook` output cells, remove trailing whitespaces and will\nblock large files to be committed. If it fails, one has to re-stage the affected files (`git add` or\n`git stage`), and re-commit.\n\n\n## `pytest` and `coverage`\n\nWe use pytest for testing the code and `coverage.py` to analyze code coverage:\n\n\n```shell\n$ # pip install pytest coverage\n$ coverage erase;\n$ coverage run --concurrency=multiprocessing -m pytest\n$ coverage combine # requried as we use multiprocessing\n$ coverage report\n$ coverage html\n```\n\n\n\n## Contributors\nThe UKIS team creates and adapts libraries which simplify the usage of satellite data. Our team\nincludes (in alphabetical order):\n* Weigand, Matthias\n\nGerman Aerospace Center (DLR)\n\n## Licenses\nThis software is licensed under the [Apache 2.0 License](LICENSE.txt).\n\nCopyright (c) 2023 German Aerospace Center (DLR) * German Remote Sensing Data Center * Department:\nGeo-Risks and Civil Security\n\n## Changelog\nSee [changelog](CHANGELOG.md).\n\n## Contributing\nThe UKIS team welcomes contributions from the community.  For more detailed information, see our\nguide on [contributing](CONTRIBUTING.md) if you're\ninterested in getting involved.\n\n## What is UKIS?\nThe DLR project Environmental and Crisis Information System (the German abbreviation is UKIS, standing for [Umwelt- und Kriseninformationssysteme](https://www.dlr.de/eoc/en/desktopdefault.aspx/tabid-5413/10560_read-21914/) aims at harmonizing the development of information systems at the German Remote Sensing Data Center\n(DFD) and setting up a framework of modularized and generalized software components.\n\nUKIS is intended to ease and standardize the process of setting up specific information systems and thus bridging the gap from EO product generation and information fusion to the delivery of products and information to end users.\n\nFurthermore, the intention is to save and broaden know-how that was and is invested and earned in the development of information systems and components in several ongoing and future DFD projects.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdlr-eoc%2Fukis_sat2h5","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdlr-eoc%2Fukis_sat2h5","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdlr-eoc%2Fukis_sat2h5/lists"}