{"id":21686218,"url":"https://github.com/pondib/openeocubes","last_synced_at":"2025-08-22T06:32:45.774Z","repository":{"id":45623059,"uuid":"510077024","full_name":"PondiB/openeocubes","owner":"PondiB","description":"A lightweight R-based RESTful service to analyze Earth Observation data cubes in the cloud.","archived":false,"fork":false,"pushed_at":"2024-08-14T09:04:13.000Z","size":7457,"stargazers_count":31,"open_issues_count":4,"forks_count":9,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-12-09T12:55:07.510Z","etag":null,"topics":["docker","earth-observation","gdalcubes","openeo","r","restful-api","stac"],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PondiB.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-03T16:30:13.000Z","updated_at":"2024-11-05T11:17:07.000Z","dependencies_parsed_at":"2023-11-18T22:36:39.751Z","dependency_job_id":"90001075-a18f-4acc-ab26-419b203cca7c","html_url":"https://github.com/PondiB/openeocubes","commit_stats":{"total_commits":202,"total_committers":7,"mean_commits":"28.857142857142858","dds":"0.47029702970297027","last_synced_commit":"cca8945712507e009ece8da567dfe22b86a94d84"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PondiB%2Fopeneocubes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PondiB%2Fopeneocubes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PondiB%2Fopeneocubes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PondiB%2Fopeneocubes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PondiB","download_url":"https://codeload.github.com/PondiB/openeocubes/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230568590,"owners_count":18246378,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","earth-observation","gdalcubes","openeo","r","restful-api","stac"],"created_at":"2024-11-25T16:28:21.767Z","updated_at":"2025-08-22T06:32:45.743Z","avatar_url":"https://github.com/PondiB.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# OpenEOcubes: openEO Compliant Lightweight R Platform for Processing Satellite Image Time Series \n\nThe service integrates STAC API (using Rstac package), the OpenEO standardized API, and data cubes concepts (using gdalcubes R package) to be a lightweight platform to enable analysis of time series satellite images via OpenEO Compliant RESTful endpoints using R, Python and JavaScript Clients. It also supports users to run their custom R functions.\n\n####  Motivation for the platform:\nThe service tries to improve on the limitations of  established EO data management platforms like Google Earth Engine and Sentinel Hub by supporting:\n* Reproducibility of Science\n* Extensibility\n* Infrastructure Replicability\n* Open Governance\n* No Need for User Management\n* User-Defined R Functions\n* Flexibility - Custom CRS,Quick Resampling of 'Large' EO Data\n\n\n![](docs/openeocubes.png)\n\nAfter processing the data , one can  download and explore on open source tools like QGIS, R, Python, etc.\n\n\n####  Future developments:\nGeospatial Machine Learning APIs for time-series EO Data:\n* ML APIs e.g. Random Forest, SVM, XGBoost, etc.\n* DL APIs e.g. TempCNN, LightTAE, etc.\n\nCurrently, PoC for ML APIs is being worked under the [Open Earth Monitor Cyberinfrastructure](https://earthmonitor.org/) EU funded project.\n## Easy Deployment from DockerHub\nAssuming you have Docker installed. This is the easiest approach.\nYou can get a hosted Docker image of the platform on DockerHub\nhttps://hub.docker.com/r/brianpondi/openeocubes\n\n### Running the container\nIt is highly recommended to deploy the service on an AWS EC2 machine that is in us-west-2 region (Oregon) as that is the data centre where the Earth Observation(EO) datasets found in AWS STAC search are stored. This enables the processing of EO data from the source so that the network latency between the platform and data is as low as possible hence cheaper. You can expose port 8000 of the EC2 instance to deploy and communicate with the service.\n```bash\ndocker run -p 8000:8000  --env AWSHOST=\u003cAWS-IPv4-ADDRESS\u003e  brianpondi/openeocubes\n```\n\nFor light tasks and processes you can host the service on pc and therefore you don't need AWS IPv4 Address\n\n```bash\ndocker run -p 8000:8000  brianpondi/openeocubes\n```\n\n## Development Notes:\n\n### Easy Deployment for LocalDevelopment\nWhile developing, you can skip rebuilding the docker container everytime. Instead you can run the server locally. \n\nYou first need to clone the repository via this command:\n\n```bash\ngit clone https://github.com/PondiB/openeocubes.git\n```\n\nthen you can change to that directory\n\n```bash\ncd openeocubes\n```\n\nYou can run the server locally by running the following command:\n```bash\nRscript startLocal.R\n\n```\nVia HTTPs with Caddy:\n\n```bash\nRscript startLocal.R \u0026 caddy run\n\n```\n\nThis will compile this Repository as a R Package and start the server.\n\n## Easy Deployment with Docker\nIf you want to change the source code then this approach is recommended.\nYou first need to clone the repository via this command:\n\n```bash\ngit clone https://github.com/PondiB/openeocubes.git\n```\n\nthen you can change to that directory\n\n```bash\ncd openeocubes\n```\n\n\n\nRun it :\n\n```bash\ndocker-compose up\n```\n\nRun in detached mode :\n\n```bash\ndocker-compose up -d\n```\n\nShutting it down:\n\n```bash\ndocker-compose down\n```\n\nForce restart  and rebuild:\n\n```bash\ndocker-compose up --build --force-recreate --no-deps -d\n```\n\nIf there are new changes on the images or Dockerfiles:\n```bash\ndocker-compose build --no-cache \u0026\u0026 docker-compose up\n\n```\n\n\n## Getting Started:\n\n### Example 1:  NDVI Script in R-Studio using OpenEO R-Client\nUsing openeo client version 1.3.0, the R scripts provided below calculate a 1-year period NDVI in a section of Amazonia in Brazil. \n\n```bash\nlibrary(openeo)\n\n# connect  to the back-end when deployed locally\n con = connect(\"http://localhost:8000\")\n# connect  to the back-end when deployed on aws\n# con = connect(\"http://\u003cAWS-IPv4-ADDRESS\u003e:8000\")\n\n# basic login with default params\nlogin(user = \"user\",\n      password = \"password\")\n\n# get the collection list\ncollections = list_collections()\n\n# to check description of a collection\ncollections$`sentinel-s2-l2a-cogs`$description\n\n# Check that required processes are available.\nprocesses = list_processes()\n\n# to check specific process e.g. ndvi\ndescribe_process(processes$ndvi)\n\n# get the process collection to use the predefined processes of the back-end\np = processes()\n\n# load the initial data collection and limit the amount of data loaded\ndatacube_init = p$load_collection(id = \"sentinel-s2-l2a-cogs\",\n                                spatial_extent = list(west = -7338335,\n                                                      south = -1027138,\n                                                      east = -7329987,\n                                                      north = -1018790,\n                                                      crs = 3857),\n                                temporal_extent = c(\"2022-01-01\", \"2022-12-31\"))\n\n# filter the data cube for the desired bands\ndatacube_filtered = p$filter_bands(data = datacube_init, bands = c(\"B04\", \"B08\"))\n\n# aggregate data cube to a year\ndatacube_agg = p$aggregate_temporal_period(data = datacube_filtered, period = \"year\", reducer = \"median\")\n\n# ndvi calculation\ndatacube_ndvi = p$ndvi(data = datacube_agg, red = \"B04\", nir = \"B08\")\n\n# supported formats\nformats = list_file_formats()\n\n# save as GeoTiff or NetCDF\nresult = p$save_result(data = datacube_ndvi, format = formats$output$GTiff)\n\n# Process and download data synchronously\nstart.time \u003c- Sys.time()\ncompute_result(graph = result, output_file = \"amazonia_2022_ndvi.tif\")\nend.time \u003c- Sys.time()\ntime.taken \u003c- end.time - start.time\ntime.taken\nprint(\"End of processes\")\n\n```\n\nVisualization of the output from the above process:\n\n![NDVI in Amazonia](docs/amazoniandvi.png)\n\n\n### Example 2:  BFAST change detection in R-Studio using OpenEO R-Client\nUsing openeo client version 1.3.0, the R scripts provided below has a user-defined function that uses bfast library to monitor changes on time series of Sentinel-2 imagery from 2016 to 2020. The study area is the region around the new Berlin-Brandenburg Tesla Gigafactory. You can run the  code on your R-studio. \n\n```bash\nlibrary(openeo)\n\n# connect  to the back-end when deployed locally\n con = connect(\"http://localhost:8000\")\n# connect  to the back-end when deployed on aws\n#con = connect(\"http://\u003cAWS-IPv4-ADDRESS\u003e:8000\")\n\n# basic login with default params\nlogin(user = \"user\",\n      password = \"password\")\n\n# get the collection list\ncollections = list_collections()\n\n# to check description of a collection\ncollections$`sentinel-s2-l2a-cogs`$description\n\n# check that required processes are available.\nprocesses = list_processes()\n\n# to check specific process e.g. filter_bands\ndescribe_process(processes$filter_bands)\n\n# get the process collection to use the predefined processes of the back-end\np = processes()\n\n# load the initial data collection and limit the amount of data loaded\ndatacube_init = p$load_collection(id = \"sentinel-s2-l2a-cogs\",\n                                  spatial_extent = list(west = 416812.2,\n                                                        south = 5803577.5,\n                                                        east = 422094.8,\n                                                        north = 5807036.1,\n                                                        crs = 32633),\n                                  temporal_extent = c(\"2016-01-01\", \"2020-12-31\"))\n\n# filter the data cube for the desired bands\ndatacube_filtered = p$filter_bands(data = datacube_init, \n                                          bands = c(\"B04\", \"B08\"))\n# aggregate data cube to monthly\ndatacube_agg = p$aggregate_temporal_period(data = datacube_filtered, \n                            period = \"month\", reducer = \"median\")\n\n# user defined R function - bfast change detection method\nchange_detection = 'function(x) {\n  knr \u003c- exp(-((x[\"B08\",]/10000)-(x[\"B04\",]/10000))^2/(2))\n  kndvi \u003c- (1-knr) / (1+knr)\n  if (all(is.na(kndvi))) {\n    return(c(NA,NA))\n  }\n    kndvi_ts = ts(kndvi, start = c(2016, 1), frequency = 12)\n    library(bfast)\n    tryCatch({\n        result = bfastmonitor(kndvi_ts, start = c(2020,1), level = 0.01)\n        return(c(result$breakpoint, result$magnitude))\n      }, error = function(x) {\n        return(c(NA,NA))\n      })\n  }'\n\n# run udf\ndatacube_udf = p$run_udf(data = datacube_agg, udf = change_detection, context =  c(\"change_date\", \"change_magnitude\"))\n\n# supported formats\nformats = list_file_formats()\n\n# save as GeoTiff or NetCDF\nresult = p$save_result(data = datacube_udf, format = formats$output$NetCDF)\n\n# Process and download data synchronously\nstart.time \u003c- Sys.time()\ncompute_result(graph = result, output_file = \"detected_changes.nc\")\nend.time \u003c- Sys.time()\ntime.taken \u003c- end.time - start.time\ntime.taken\nprint(\"End of processes\")\n\n```\n\nVisualization of the output from the above process:\n\n![bfast change detection](docs/change_over_period.png)\n\n\n## Citation:\n\nIf you use this codebase, or otherwise found our work valuable, please cite:\n\n```bibtex\n@Article{Pondi2024,\nauthor={Pondi, Brian\nand Appel, Marius\nand Pebesma, Edzer},\ntitle={OpenEOcubes: an open-source and lightweight R-based RESTful web service for analyzing earth observation data cubes},\njournal={Earth Science Informatics},\nyear={2024},\nmonth={Feb},\nday={19},\nabstract={In recent decades, Earth Observation (EO) systems have seen remarkable technological advancements, leading to a surge in Earth-orbiting satellites capturing EO data. Cloud-based storage solutions have been adopted to manage the increasing data volume. Although numerous EO data management and analysis platforms have emerged to accommodate this growth, many suffer from limitations like closed-source software, leading to platform lock-in and restricted functionalities, restricting the scientific community from conducting open and reproducible research. To tackle these issues, we present OpenEOcubes, a lightweight EO data cubes analysis service that embraces open-source tools, standardized APIs, and containerized deployment, we demonstrate the service's capabilities in two user scenarios: performing vegetation analysis in Amazonia, Brazil for one year, and detecting changes in a forested area in Brandenburg, Germany based on five years of EO data.OpenEOcubes is an easy-to-deploy service that empowers the scientific community to reproduce small and medium-sized EO scientific analysis while aggregating over a potentially huge amount of data. It enables the extension of functionalities and validation of analysis carried out on different EO data processing platforms.},\nissn={1865-0481},\ndoi={10.1007/s12145-024-01249-y},\nurl={https://doi.org/10.1007/s12145-024-01249-y}\n}\n\n\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpondib%2Fopeneocubes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpondib%2Fopeneocubes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpondib%2Fopeneocubes/lists"}