{"id":37465618,"url":"https://github.com/ollinevalainen/satellitetools","last_synced_at":"2026-01-16T07:09:25.887Z","repository":{"id":44760884,"uuid":"270676132","full_name":"ollinevalainen/satellitetools","owner":"ollinevalainen","description":"Get Sentinel-2 data from Google Earth Engine or from cloud-optimized geotiffs in AWS Open data registry maintained by Element84, and compute biophysical parameters using SNAP Biophysical processor algorithm.","archived":false,"fork":false,"pushed_at":"2026-01-13T16:53:27.000Z","size":4643,"stargazers_count":21,"open_issues_count":2,"forks_count":9,"subscribers_count":4,"default_branch":"master","last_synced_at":"2026-01-13T18:40:08.705Z","etag":null,"topics":["cloud-optimized-geotiff","google-earth-engine","lai","python","remote-sensing","satellite-imagery","sentinel-2","vegetation","vegetation-index"],"latest_commit_sha":null,"homepage":"https://satellitetools.readthedocs.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ollinevalainen.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-06-08T13:19:11.000Z","updated_at":"2026-01-13T16:53:30.000Z","dependencies_parsed_at":"2024-01-04T14:47:27.169Z","dependency_job_id":"c5e14307-a114-438a-a83e-238fa3e0d2fc","html_url":"https://github.com/ollinevalainen/satellitetools","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/ollinevalainen/satellitetools","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ollinevalainen%2Fsatellitetools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ollinevalainen%2Fsatellitetools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ollinevalainen%2Fsatellitetools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ollinevalainen%2Fsatellitetools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ollinevalainen","download_url":"https://codeload.github.com/ollinevalainen/satellitetools/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ollinevalainen%2Fsatellitetools/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28478017,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T06:30:42.265Z","status":"ssl_error","status_checked_at":"2026-01-16T06:30:16.248Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-optimized-geotiff","google-earth-engine","lai","python","remote-sensing","satellite-imagery","sentinel-2","vegetation","vegetation-index"],"created_at":"2026-01-16T07:09:25.238Z","updated_at":"2026-01-16T07:09:25.869Z","avatar_url":"https://github.com/ollinevalainen.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![DOI](https://zenodo.org/badge/270676132.svg)](https://zenodo.org/badge/latestdoi/270676132)\n\n# satellitetools 🛰️ \nThis package provides methods to get Sentinel-2 L2A data from Google Earth Engine \n(\u003chttps://developers.google.com/earth-engine/\u003e) or from cloud-optimized geotiffs in AWS Open data registry \nmaintained by Element84 (\u003chttps://registry.opendata.aws/sentinel-2-l2a-cogs/\u003e, \u003chttps://github.com/Element84/earth-search\u003e).\n\nThis package has been mainly used in agricultural projects at the Climate System Research unit at \nthe Finnish Meteorological Institute. The package is at the moment under constant development. \nApologies for the lack of documentation at the moment. Please feel \nfree to contact me in case you want more information about this package.\n\n Package also includes python implementation of ESA's SNAP Biophysical processor v1 \n which can be used to compute biophysical parameters, such as leaf area index (LAI) (Original Java code in SNAP [here](https://github.com/senbox-org/s2tbx/tree/master/s2tbx-biophysical/src/main/java/org/esa/s2tbx/biophysical). Algorithm documentation: \u003chttp://step.esa.int/docs/extra/ATBD_S2ToolBox_L2B_V1.1.pdf\u003e.\n\n\nDocumentation: \u003chttps://satellitetools.readthedocs.io/\u003e\n\nSource code at: \u003chttps://github.com/ollinevalainen/satellitetools\u003e\n\nCiting: If you use this package in your research, I would appeaciate if you cite the relevant version from Zenodo: \u003chttps://doi.org/10.5281/zenodo.5993291\u003e\n\nAuthor: Olli Nevalainen, Finnish Meteorological Institute.\n\n\n\n## Installation ##\n\n**From PyPi:**\n```console\npip install satellitetools\n```\n\n**From GitHub:**\n```console\npip install git+https://github.com/ollinevalainen/satellitetools.git\n```\n\n**Development branch:**\n```console\npip install git+https://github.com/ollinevalainen/satellitetools.git@develop\n```\n\n## About Google Earth Engine ##\nEarth Engine must bee authenticated and initialized by the user.\nGuides to authentication: \u003chttps://developers.google.com/earth-engine/guides/auth\u003e\n\nRemember GEE's licence terms.\n\n## About AWS cloud-optimized geotiffs ##\nCurrently (2024-10-23) 2022 data is missing still from the sentinel-2-c1-l2a collection\nbecause ESA has not yet processed them. 2022 is instead fetched from the sentinel-2-l2a\ncollection. Follow discussion and issues related to this at: \u003chttps://github.com/Element84/earth-search/issues\u003e.\n\n## Warning ##\nThe biophysical processor implementation in this package does not currently use the \nconvex hull check (see the Java code and ATBD) and does not have as extensive \ninput/output validity flagging as the original version in SNAP.\n\n\n## Usage examples ##\n\n\n**Define area of interest and request parameters:**\n```python\nfrom shapely.geometry import Polygon\nimport satellitetools as sattools\nfrom satellitetools.common.sentinel2 import S2_BANDS_10_20_GEE, S2_FILTER1\n\n\nqvidja_ec_field_geom = [\n    [22.3913931, 60.295311],\n    [22.3917056, 60.2951721],\n    [22.3922131, 60.2949717],\n    [22.3927016, 60.2948124],\n    [22.3932251, 60.2946874],\n    [22.3931117, 60.2946416],\n    [22.3926039, 60.2944037],\n    [22.3920127, 60.2941585],\n    [22.3918447, 60.2940601],\n    [22.391413, 60.2937852],\n    [22.3908102, 60.2935286],\n    [22.390173, 60.2933897],\n    [22.389483, 60.2933106],\n    [22.3890777, 60.293541],\n    [22.3891442, 60.2936358],\n    [22.3889863, 60.2940313],\n    [22.3892131, 60.2941537],\n    [22.3895462, 60.2942468],\n    [22.3899066, 60.2944289],\n    [22.3903881, 60.2946329],\n    [22.3904738, 60.2948121],\n    [22.3913931, 60.295311],\n]\n\nqvidja_polygon = Polygon(qvidja_ec_field_geom)\naoi = sattools.AOI(\"qvidja_ec\", qvidja_polygon, \"EPSG:4326\")\ndatestart = \"2023-06-01\"\ndateend = \"2023-06-30\"\nbands = sattools.S2Band.get_10m_to_20m_bands()\n\n```\n\n**GEE:**\n\nYou need to initialize (and possibly authenticate) Google Earth Engine and define GEE \ndata source.\n```python\nimport ee\nee.Initialize(project=\"your-ee-project-name\")\ndatasource = sattools.DataSource.GEE\n```\n\n\n**AWS:**\n\nDefine AWS data source.\n```python\ndatasource = sattools.DataSource.AWS\n```\n\n**Evaluate quality and get the data using a wrapper function:**\n```python\nreq_params = sattools.Sentinel2RequestParams(\n    datestart, dateend, datasource, bands\n)\n\ndf_quality_information, ds_s2_data = sattools.wrappers.get_s2_qi_and_data(\n    aoi=aoi,\n    req_params=req_params,\n    qi_threshold=0.02,\n    qi_filter=S2_FILTER1,\n)\n# df_quality_information will include quality information for all available Sentinel-2 images.\n\n# ds_s2_data will include Sentinel-2 data for the requested bands and metadata for dates passing quality filter (see below more on the quality evaluation).\n```\n**Alternatively without the wrapper:**\n\nThis way you can access the data collection class with some extra metadata and with AWS\nthe source data item. With AWS data source you can speed-up process by concurrently fetching \nmultiple images with multiprocessing parameter.\n```python\nif req_params.datasource == DataSource.GEE:\n    s2_data_collection = GEESentinel2DataCollection(aoi, req_params)\nif req_params.datasource == DataSource.AWS:\n    import os\n    number_of_processes = os.cpu_count() - 1 # or something else\n    s2_data_collection = AWSSentinel2DataCollection(aoi, req_params, multiprocessing=number_of_processes)\n    s2_data_collection.search_s2_items()\n\ns2_data_collection.get_quality_info()\nqi_threshold=0.02 # without specifying default 0 will be used\nqi_filter=S2_FILTER1\ns2_data_collection.filter_s2_items(qi_threshold, qi_filter)\ns2_data_collection.get_s2_data()\ns2_data_collection.data_to_xarray()\n\ndf_quality_information = s2_data_collection.quality_information\nds_s2_data = s2_data_collection.xr_dataset\n```\n\n**Calculate vegetation variables:**\n```python\n# Example to calculate LAI and NDVI.\nvariable = sattools.biophys.BiophysVariable.LAI\nds_s2_data = sattools.biophys.run_snap_biophys(ds_s2_data, variable)\nvi = sattools.biophys.VegetationIndex.NDVI\nds_s2_data = sattools.biophys.compute_vegetation_index(ds_s2_data, vi)\n# Both LAI and NDVI are now included in the dataset with variables names lai and ndvi.\n\n# Other biophysical variables and vegetation indices also available:\nclass VegetationIndex(str, Enum):\n    NDVI = \"ndvi\"\n    CI_RED_EDGE = \"ci_red_edge\"\n    GCC = \"gcc\"\n\n\nclass BiophysVariable(str, Enum):\n    FAPAR = \"fapar\"\n    FCOVER = \"fcover\"\n    LAI = \"lai\"\n    LAI_Cab = \"lai_cab\"\n    LAI_Cw = \"lai_cw\"\n```\n\n## Testing\nTests have been written for pytest.\n\nIf you need to specify Google Earth Engine project for ee.Initialize(), for test you \nneed to do it by having \"EE_PROJECT_PYTEST\" environment variable set. For example:\n```console\nexport EE_PROJECT_PYTEST=your-ee-project\n```\n\n## About the satellite data filtering process ##\n\nThe satellite data is filtered based on the information available in the scene \nclassification band (SCL) of the Sentinel-2 Level 2A products. \nIn the SCL band every pixel is classified into one of the following classes: \n\n```python\nclass SCLClass(Enum):\n    \"\"\"Sentinel-2 Scene Classification Layer (SCL) classes.\"\"\"\n    NODATA = 0\n    SATURATED_DEFECTIVE = 1\n    DARK_FEATURE_SHADOW = 2\n    CLOUD_SHADOW = 3\n    VEGETATION = 4\n    NOT_VEGETATED = 5\n    WATER = 6\n    UNCLASSIFIED = 7\n    CLOUD_MEDIUM_PROBA = 8\n    CLOUD_HIGH_PROBA = 9\n    THIN_CIRRUS = 10\n    SNOW_ICE = 11\n```\n\nThe percentage of each of those classes within the polygon/area-of-interest(AOI) is \ncalculated from SCL data. This information is/can be used to to \nfilter the dates for which the actual data is requested from GEE or AWS. \nThere's two parameters, qi_threhold and qi_filter, which are used to filter the data. \nThe qi_filter is a list of the (unwanted) classes whose percentages within the AOI is \nsummed. Currently, qi_filter is by default this:\n\n```python\nS2_FILTER1 = [\n    \"NODATA\",\n    \"SATURATED_DEFECTIVE\",\n    \"CLOUD_SHADOW\",\n    \"UNCLASSIFIED\",\n    \"CLOUD_MEDIUM_PROBA\",\n    \"CLOUD_HIGH_PROBA\",\n    \"THIN_CIRRUS\",\n    \"SNOW_ICE\",\n]\n```\nThese classes are considered unwanted and user can change the classes included in that\nlist by the qi_filter parameter. If the sum of the percentages of the classes defined \nin the qi_filter is less than the qi_threshold, the data is considered good/acceptable. \nI have been using the 0.02 which I have empirically tested and found to be ok for our  \npurposes.\n\nThe SCL band provided in the S2 level-2A products is automatically generated and it has \nerrors/misclassifications in it. This affects also the accuracy of the filtering \nprocesss and some bad data acquisition dates might pass the filter.\n\n\n## TODO ##\n**Realistic:**\n\n- [ ] Include example notebook in the documentation\n- [ ] Examples to docstrings\n- [ ] Improve tests\n- [ ] Add coverage and docs bulding to nox sessions\n- [ ] Dcoument branching strategy and add contributing guidelines\n- [ ] Refactor ee_* functions in gee.py. They have a lot of repetition.\n- [ ] Calculate vegetation indices directly in GEE cloud and return only timeseries\n\n**Wishful thinking:**\n- [ ] Point-based AOI (currently AOI must be a polygon)\n- [ ] Add Landsat data support\n- [ ] Add Sentinel-1 data support\n- [ ] Large spatial scale data retrieval from AWS: yield daily xr_datasets\n    - currently all data is fetched first to memory\n\n## History of major versions ##\n\n### v2.0.0 ###\n\nThe package ended-up being quite useful in multiple research projects, so development of the package has been continuous. This release is a milestone for years of small changes and recent major refactoring of code during past two months. But now develop branch was finally merged to master.\n\nIn the future, the plan is to publish releases more frequently when possible developments in the develop branch gets merged to master branch.\n\nToo much has changed compared to the first release, but here's summary of changes respect to the previous develop branch before major refactoring done during Sep-Dec 2024:\n\n **Breaking changes:**:warning:\n\n* Renamed xrtools.py to timeseries.py\n* RequestParams class is now Sentinel2RequestParams in sentinel2 submodule\n* Restructured files and removed nesting. For example gee imported as satellitetools.gee instead of satellitetools.gee.gee\n* The quality information dataframe doesn't have anymore `\"Date\"` column, but instead index named `\"acquisition_time\"` as UTC aware timestamp (from `pd.to_datetime`).\n\n**New features:** 🔧 \n\n* Refactored and made codes more object-oriented and modular:\n* There's now parent classes `Sentinel2DataCollection`, `Sentinel2Item` and `Sentinel2Metadata` which have datasource specific child classes:\n    - `GEESentinel2DataCollection`\n    - `AWSSentinel2DataCollection`, `AWSSentinel2Item`, `AWSSentinel2Metadata`\n* The parent classes have methods that are common for both data sources and the child classes have methods that are specific to the data source.\n* Improved handling of Sentinel-2 bands and scene classification classes with `S2Band` and `SCLClass` classes\n* Biophysical processor is now a class `SNAPBiophysProcessor`, also the biophysical variables and vegetation indices are now Enum classes.\n* Enabled easier importing and access of classes and submodules. For example, you can define the data source with `satellitetools.DataSource.GEE` and the bands with `satellitetools.S2Band.B4`.\n* Added tests for pytest.\n* Improved docstrings, added examples and updated README.\n* Documentation using sphinx\n* Testing for python versions 3.10, 3.11 and 3.12 using nox\n* Changed from print statements to logging.\n\n## v1.0.0\nThis release is currently (7 February 2022) in use in Field Observatory (https://www.fieldobservatory.org/) and cited in a scientific article \"Towards agricultural soil carbon monitoring, reporting and verification through Field Observatory Network (FiON)\" that describes Field Observatory and has been accepted for publishing in EGU's Geoscientific Instrumentation, Methods and Data Systems journal (https://doi.org/10.5194/gi-2021-21).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Follinevalainen%2Fsatellitetools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Follinevalainen%2Fsatellitetools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Follinevalainen%2Fsatellitetools/lists"}