{"id":18819094,"url":"https://github.com/geoscienceaustralia/wagl","last_synced_at":"2025-07-29T06:09:23.016Z","repository":{"id":9206319,"uuid":"11015951","full_name":"GeoscienceAustralia/wagl","owner":"GeoscienceAustralia","description":"Python library for standardising satellite imagery into an Analysis Ready Data (ARD) form","archived":false,"fork":false,"pushed_at":"2024-09-30T16:29:42.000Z","size":17837,"stargazers_count":31,"open_issues_count":15,"forks_count":7,"subscribers_count":29,"default_branch":"develop","last_synced_at":"2025-04-13T23:49:44.045Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GeoscienceAustralia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2013-06-28T04:44:42.000Z","updated_at":"2025-03-17T14:52:56.000Z","dependencies_parsed_at":"2023-01-13T15:30:13.267Z","dependency_job_id":"18f79f0d-65b4-4bf0-ba82-5f44e883c100","html_url":"https://github.com/GeoscienceAustralia/wagl","commit_stats":null,"previous_names":[],"tags_count":93,"template":false,"template_full_name":null,"purl":"pkg:github/GeoscienceAustralia/wagl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GeoscienceAustralia%2Fwagl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GeoscienceAustralia%2Fwagl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GeoscienceAustralia%2Fwagl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GeoscienceAustralia%2Fwagl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GeoscienceAustralia","download_url":"https://codeload.github.com/GeoscienceAustralia/wagl/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GeoscienceAustralia%2Fwagl/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267639025,"owners_count":24119768,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-29T02:00:12.549Z","response_time":2574,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T00:20:15.866Z","updated_at":"2025-07-29T06:09:22.994Z","avatar_url":"https://github.com/GeoscienceAustralia.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# wagl\n------\n\n\nwagl is a Python package for producing standarised imagery in the form of:\n\n* Nadir Bi-directional Reflectance Distribution Function Adjusted Reflectance (NBAR)\n* NBART; NBAR with Terrain Illumination correction\n* Surface Brightness Temperature\n* Pixel Quality (per pixel metadata)\n\nThe luigi task workflow for producing NBAR for a Landsat 5TM scene is given below.\n\n![](docs/source/diagrams/luigi-task-visualiser-reduced.png)\n\n## Supported Satellites and Sensors\n-----------------------------------\n* Landsat 5 TM\n* Landsat 7 ETM\n* Landsat 8 OLI\n* Landsat 8 TIRS\n* Sentinel-2a\n\n## Requirements\n---------------\n* [luigi](https://github.com/spotify/luigi)\n* [numpy](https://github.com/numpy/numpy)\n* [scipy](https://github.com/scipy/scipy)\n* [numexpr](https://github.com/pydata/numexpr)\n* [pyephem](http://rhodesmill.org/pyephem/)\n* [proj](https://github.com/OSGeo/proj.4)\n* [h5py](https://github.com/h5py/h5py)\n* [tables](https://github.com/PyTables/PyTables)\n* [pandas](https://github.com/pandas-dev/pandas)\n* [scikit-image](https://github.com/scikit-image/scikit-image)\n* [GDAL](https://github.com/OSGeo/gdal)\n* [rasterio](https://github.com/mapbox/rasterio)\n* [fiona](https://github.com/Toblerity/Fiona)\n* [shapely](https://github.com/Toblerity/Shapely)\n* [geopandas](https://github.com/geopandas/geopandas)\n* [pyyaml](https://github.com/yaml/pyyaml)\n* [attrs](https://github.com/python-attrs/attrs)\n\n## Installation\n---------------\n\n### wagl Package\nThe wagl pacakage can be installed via:\n\n`$ python setup.py install --prefix=\u003cprefix\u003e`\n\n### Additional HDF5 compression filters (optional)\nAdditional compression filters can be used via HDF5's\n[dynamically loaded filters](https://support.hdfgroup.org/HDF5/doc/Advanced/DynamicallyLoadedFilters/HDF5DynamicallyLoadedFilters.pdf).\nEssentially the filter needs to be compiled against the HDF5 library, and\ninstalled into HDF5's plugin path, or a path of your choosing, and set the\nHDF5_PLUGIN_PATH environment variable. The filters are then automatically\naccessible by HDF5 via the [integer code](https://support.hdfgroup.org/services/contributions.html)\nassigned to the filter.\n\n#### Mafisc compression filter\nMafisc combines both a bitshuffling filter and lzma compression filter in order\nto get the best compression possible at the cost of lower compression speeds.\nTo install the `mafisc` compression filter, follow these [instructions](https://wr.informatik.uni-hamburg.de/research/projects/icomex/mafisc).\n\n#### Bitshuffle\nThe [bitshuffle filter](https://github.com/kiyo-masui/bitshuffle) can be installed\nfrom source, or conda via the supplied [conda recipe](https://github.com/kiyo-masui/bitshuffle/tree/master/conda-recipe).\nIt utilises a bitshuffling filter on top of either a lz4 or lzf compression filter.\n\n## Basic command line useage\n--------------------------\nUsing the [local scheduler](http://luigi.readthedocs.io/en/stable/command_line.html):\n\n    $ luigi --module wagl.multifile_workflow ARD --workflow NBAR --level1-list scenes.txt --outdir /some/path --local-scheduler --workers 4\n\nUsing the [central scheduler](http://luigi.readthedocs.io/en/stable/central_scheduler.html):\n\n    $ luigid --background --pidfile \u003cPATH_TO_PIDFILE\u003e --logdir \u003cPATH_TO_LOGDIR\u003e --state-path \u003cPATH_TO_STATEFILE\u003e\n\n    $ luigi --module wagl.multifile_workflow ARD --level1-list scenes.txt --workflow STANDARD --outdir /some/path --workers 4\n\n    $ luigi --module wagl.multifile_workflow ARD --level1-list scenes.txt --workflow NBAR --outdir /some/path --workers 4\n\n    $ luigi --module wagl.multifile_workflow ARD --level1-list scenes.txt --workflow SBT --outdir /some/path --workers 4\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeoscienceaustralia%2Fwagl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgeoscienceaustralia%2Fwagl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeoscienceaustralia%2Fwagl/lists"}