{"id":25666484,"url":"https://github.com/parvvaresh/etl-gee","last_synced_at":"2025-04-22T19:12:07.466Z","repository":{"id":214096104,"uuid":"735672197","full_name":"parvvaresh/ETL-gee","owner":"parvvaresh","description":null,"archived":false,"fork":false,"pushed_at":"2024-11-24T19:34:37.000Z","size":47,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-22T19:11:58.974Z","etag":null,"topics":["etl-pipeline","get-data","google-earth-engine","satellite-imagery"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/parvvaresh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-25T19:08:22.000Z","updated_at":"2025-04-04T19:36:01.000Z","dependencies_parsed_at":"2023-12-25T20:57:47.671Z","dependency_job_id":"407b6e28-b855-4027-9223-7a5a2149e41d","html_url":"https://github.com/parvvaresh/ETL-gee","commit_stats":null,"previous_names":["parvvaresh/genetic-algorithm-for-string-guessing","parvvaresh/google-earth-engine","parvvaresh/etl-gee"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parvvaresh%2FETL-gee","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parvvaresh%2FETL-gee/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parvvaresh%2FETL-gee/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/parvvaresh%2FETL-gee/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/parvvaresh","download_url":"https://codeload.github.com/parvvaresh/ETL-gee/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250306640,"owners_count":21408926,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["etl-pipeline","get-data","google-earth-engine","satellite-imagery"],"created_at":"2025-02-24T08:30:27.922Z","updated_at":"2025-04-22T19:12:07.449Z","avatar_url":"https://github.com/parvvaresh.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Google Earth Engine\n\n# 1. Get Data\n\n## Overview\n\nThe `gee` library is a Python package designed for processing and analyzing Sentinel-1 and Sentinel-2 satellite imagery using Google Earth Engine (GEE). It allows for the loading, processing, and combining of Sentinel-1 and Sentinel-2 data, as well as the calculation of various vegetation indices. Additionally, it supports exporting the processed data to Google Drive.\n\n## Installation\n\nBefore using the `gee` library, you need to set up the Google Earth Engine Python API. Follow the official installation guide [here](https://developers.google.com/earth-engine/guides/python_install).\n\n## Usage\n\nHere's a step-by-step guide on how to use the `gee` library.\n\n### 1. Import the Library\n\n```python\nimport ee\nfrom gee import gee\n\n# Initialize the Earth Engine library\nee.Initialize()\nee.Initialize(project='name of project')\n```\n\n\n### 2. Clone form github\n\nThis code can only be used in Google Colab\n\n```python\n!git clone https://github.com/parvvaresh/google-earth-engine\n%cd google-earth-engine\n\nfrom get_data.gee import gee\n```\n### 3. Define the Area of Interest (AOI)\n\nYou need to define your area of interest (AOI) as an `ee.Geometry`. For example, to define a rectangular AOI:\n\n```python\naoi = ee.Geometry.Rectangle([xmin, ymin, xmax, ymax])\n```\n\n### 4. Create an Instance of the `gee` Class\n\n```python\ngee_instance = gee(aoi)\n```\n\nOptionally, you can provide a `table_clip` parameter if you want to clip the results to a specific geometry.\n\n### 5. Run the Data Pipeline\n\nCall the `pipeline_data` method with the required parameters to process and export the data:\n\n```python\ngee_instance.pipeline_data(\n    start_date='YYYY-MM-DD',  # Start date of the data collection period\n    end_date='YYYY-MM-DD',    # End date of the data collection period\n    name_file='exported_data',  # Name of the exported file\n    name_folder='GEE_exports'  # Name of the folder in Google Drive where the file will be saved\n)\n```\n\n### Example Usage\n\n```python\nimport ee\nfrom gee import gee\n\n# Initialize the Earth Engine library\nee.Initialize()\nee.Initialize(project='name of project')\n\n\n!git clone https://github.com/parvvaresh/google-earth-engine\n%cd google-earth-engine\n\nfrom get_data.gee import gee\n\n# Define your AOI (example coordinates)\naoi = ee.Geometry.Rectangle([-10, 35, 10, 45])\n\n# Create an instance of the gee class\ngee_instance = gee(aoi)\n\n# Run the data pipeline\ngee_instance.pipeline_data(\n    start_date='2023-01-01',\n    end_date='2023-01-31',\n    name_file='sentinel_data',\n    name_folder='GEE_exports'\n)\n```\n\n## Methods\n\n### `_load_Sentinel1(self, start_date: str, end_date: str) -\u003e None`\n\nLoads the Sentinel-1 ImageCollection within the specified date range and AOI.\n\n### `_process_Sentinel1(self) -\u003e None`\n\nProcesses the loaded Sentinel-1 data, including filtering and creating mosaics based on ascending and descending orbit passes.\n\n### `_load_Sentinel2(self, start_date: str, end_date: str, cloudy_pixel: int) -\u003e None`\n\nLoads the Sentinel-2 ImageCollection within the specified date range, AOI, and cloud cover percentage.\n\n### `_process_Sentinel2(self, interval: int, start_date: str, end_date: str) -\u003e None`\n\nProcesses the loaded Sentinel-2 data, creating composites at specified intervals and calculating NDVI, EVI, and SAVI indices.\n\n### `_combine_sentinel1_sentinel2(self) -\u003e None`\n\nCombines the processed Sentinel-1 and Sentinel-2 data into a single dataset and clips it if a `table_clip` is provided.\n\n### `_export_data(self, name_file: str, name_folder: str) -\u003e None`\n\nExports the processed and combined data to Google Drive as a CSV file.\n\n### `pipeline_data(self, start_date: str, end_date: str, name_file: str, name_folder: str) -\u003e None`\n\nRuns the entire data pipeline, from loading and processing Sentinel-1 and Sentinel-2 data to exporting the results.\n\n## Notes\n\n- Ensure that you have sufficient permissions and quota in your Google Earth Engine account to run the processing tasks.\n- The export task may take some time depending on the size of the AOI and the date range specified.\n\n\n# 2. Convert to csv\n\n## License\n\nThis code is provided under the MIT License. Feel free to use and modify it as needed.\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparvvaresh%2Fetl-gee","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparvvaresh%2Fetl-gee","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparvvaresh%2Fetl-gee/lists"}