{"id":13585701,"url":"https://github.com/samapriya/geeup","last_synced_at":"2025-12-28T09:35:30.286Z","repository":{"id":44566982,"uuid":"144343870","full_name":"samapriya/geeup","owner":"samapriya","description":"Simple CLI for Google Earth Engine Uploads","archived":false,"fork":false,"pushed_at":"2024-10-03T01:26:59.000Z","size":8448,"stargazers_count":126,"open_issues_count":4,"forks_count":26,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-28T01:08:00.175Z","etag":null,"topics":["batch-upload","cli","command-line-tool","earth-engine-uploads","gee-upload","google-earth-engine"],"latest_commit_sha":null,"homepage":"https://geeup.geetools.xyz","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/samapriya.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"samapriya","custom":["buymeacoffee.com/samapriya"]}},"created_at":"2018-08-11T00:39:33.000Z","updated_at":"2025-03-24T03:30:44.000Z","dependencies_parsed_at":"2023-11-29T06:36:01.744Z","dependency_job_id":"2443465a-f8e7-47d3-8f55-c993c0210bea","html_url":"https://github.com/samapriya/geeup","commit_stats":{"total_commits":222,"total_committers":1,"mean_commits":222.0,"dds":0.0,"last_synced_commit":"f76ac804e03f33073f283b95781a2b8e6aea35be"},"previous_names":[],"tags_count":34,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samapriya%2Fgeeup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samapriya%2Fgeeup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samapriya%2Fgeeup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samapriya%2Fgeeup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/samapriya","download_url":"https://codeload.github.com/samapriya/geeup/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247107820,"owners_count":20884795,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["batch-upload","cli","command-line-tool","earth-engine-uploads","gee-upload","google-earth-engine"],"created_at":"2024-08-01T15:05:05.693Z","updated_at":"2025-12-28T09:35:30.279Z","avatar_url":"https://github.com/samapriya.png","language":"Python","funding_links":["https://github.com/sponsors/samapriya","buymeacoffee.com/samapriya"],"categories":["Python","Python API","Geospatial Library","Google Earth Engine","Earth Engine"],"sub_categories":["Packages","Python","Testing your code"],"readme":"# [geeup: Simple CLI for Earth Engine Uploads](https://geeup.geetools.xyz)\n\n[![LinkedIn](https://img.shields.io/badge/LinkedIn-0077B5?style=plastic\u0026logo=linkedin\u0026logoColor=white)](https://www.linkedin.com/in/samapriya/)\n[![Medium](https://img.shields.io/badge/Medium-12100E?style=flat\u0026logo=medium\u0026logoColor=white)](https://medium.com/@samapriyaroy)\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.18073520.svg)](https://doi.org/10.5281/zenodo.18073520)\n[![PyPI version](https://badge.fury.io/py/geeup.svg)](https://badge.fury.io/py/geeup)\n[![Downloads](https://static.pepy.tech/badge/geeup)](https://pepy.tech/project/geeup)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n![CI geeup](https://github.com/samapriya/geeup/workflows/CI%20geeup/badge.svg)\n[![](https://img.shields.io/static/v1?label=Sponsor\u0026message=%E2%9D%A4\u0026logo=GitHub\u0026color=%23fe8e86)](https://github.com/sponsors/samapriya)\n\n**geeup** is a comprehensive command-line tool designed to simplify data preparation and batch uploading to Google Earth Engine. Handle images, shapefiles, and metadata with retry logic, parallel processing, and enhanced path normalization.\n\n## Citation\n\nIf you use this project, please star and cite it as below:\n\n```\nSamapriya Roy. (2025). samapriya/geeup: geeup: Simple CLI for Earth Engine Uploads (2.0.0).\nZenodo. https://doi.org/10.5281/zenodo.18073520\n```\n\n## Table of Contents\n\n- [geeup: Simple CLI for Earth Engine Uploads](#geeup-simple-cli-for-earth-engine-uploads)\n  - [Citation](#citation)\n  - [Table of Contents](#table-of-contents)\n  - [Installation](#installation)\n    - [Method 1: PyPI (Recommended)](#method-1-pypi-recommended)\n    - [Method 2: From Source](#method-2-from-source)\n  - [Getting Started](#getting-started)\n  - [Authentication](#authentication)\n    - [Cookie Setup](#cookie-setup)\n    - [Service Account Configuration](#service-account-configuration)\n  - [Data Preparation](#data-preparation)\n    - [Rename Files](#rename-files)\n    - [Zip Shapefiles](#zip-shapefiles)\n    - [Generate Metadata](#generate-metadata)\n  - [Batch Upload](#batch-upload)\n    - [Upload Images](#upload-images)\n    - [Upload Tables (Tabup)](#upload-tables-tabup)\n  - [Monitoring](#monitoring)\n    - [Check Quota](#check-quota)\n    - [Task Status](#task-status)\n    - [Cancel Tasks](#cancel-tasks)\n  - [Utilities](#utilities)\n    - [Delete Assets](#delete-assets)\n    - [Open Documentation](#open-documentation)\n  - [Full Documentation](#full-documentation)\n  - [Changelog](#changelog)\n    - [Version 2.0.0](#version-200)\n  - [Contributing](#contributing)\n  - [License](#license)\n  - [Acknowledgments](#acknowledgments)\n\n## Installation\n\n### Method 1: PyPI (Recommended)\n\n```bash\npip install geeup\n```\n\n### Method 2: From Source\n\n```bash\ngit clone https://github.com/samapriya/geeup.git\ncd geeup\npip install -e .\n```\n\n**Requirements:**\n- Python 3.7+\n- Earth Engine Python API\n- Authenticated Earth Engine account (`earthengine authenticate`)\n\n## Getting Started\n\nView all available commands:\n\n```bash\ngeeup -h\n```\n\nMain interface output:\n\n```\nusage: geeup [-h] {readme,quota,auth,rename,zipshape,getmeta,cookie_setup,upload,tabup,tasks,cancel,delete} ...\n\nSimple Client for Earth Engine Uploads with Enhanced UI and Service Account Support\n\npositional arguments:\n  {readme,quota,auth,rename,zipshape,getmeta,cookie_setup,upload,tabup,tasks,cancel,delete}\n    readme              Open the geeup documentation page\n    quota               Print Earth Engine storage and asset count quota with visual progress bars\n    auth                Configure service account authentication\n    rename              Rename files to adhere to EE naming rules with interactive preview\n    zipshape            Zip all shapefiles and subsidiary files in folder with progress tracking\n    getmeta             Create generalized metadata for rasters in folder with progress tracking\n    cookie_setup        Setup cookies to be used for upload\n    upload              Batch Image Uploader for uploading TIF files to a GEE collection\n    tabup               Batch Table Uploader for uploading shapefiles/CSVs to a GEE folder\n    tasks               Query current task status with rich formatting [completed, running, ready, failed, cancelled]\n    cancel              Cancel all, running, pending tasks or specific task ID with progress tracking\n    delete              Delete collection and all items inside recursively\n```\n\n## Authentication\n\n### Cookie Setup\n\nThe `cookie_setup` command configures authentication cookies required for uploads. This uses the [Copy Cookies Chrome extension](https://chrome.google.com/webstore/detail/copy-cookies/jcbpglbplpblnagieibnemmkiamekcdg/related).\n\n**Setup Steps:**\n\n1. Install the Copy Cookies extension in Chrome\n2. Log into [code.earthengine.google.com](https://code.earthengine.google.com)\n3. Click the extension icon to copy cookies\n4. Run the setup command and paste when prompted:\n\n```bash\ngeeup cookie_setup\n```\n\n**Important Notes:**\n- Use a fresh browser window (not incognito)\n- Clear cookies and retry if you get \"Unable to read\" errors\n- Ensure cookies match the account initialized with `earthengine authenticate`\n\n**Platform-Specific Instructions:**\n\nFor Bash users experiencing paste limitations:\n```bash\nstty -icanon\ngeeup cookie_setup\nstty icanon\n```\n\nFor Mac users, you may need to change your default shell from `/bin/zsh` to `/bin/sh`.\n\n### Service Account Configuration\n\nThe `auth` command manages service account authentication for automated workflows.\n\n**Initialize with service account:**\n```bash\ngeeup auth --cred \"/path/to/service-account.json\"\n```\n\n**Check authentication status:**\n```bash\ngeeup auth --status\n```\n\n**Remove credentials:**\n```bash\ngeeup auth --remove\n```\n\n**Arguments:**\n- `--cred`: Path to service account JSON credentials file (optional)\n- `--status`: Show current authentication status (optional)\n- `--remove`: Remove stored service account credentials (optional)\n\n## Data Preparation\n\n### Rename Files\n\nSanitize filenames to adhere to Earth Engine naming conventions. Removes spaces and special characters.\n\n```bash\ngeeup rename --input \"./raw_data\" --batch\n```\n\n**Arguments:**\n- `--input`: Path to directory with files to rename (required)\n- `--batch`: Skip confirmation and rename all files automatically (optional)\n\n**Interactive Mode:**\nBy default, shows a preview of changes and asks for confirmation before renaming.\n\n### Zip Shapefiles\n\nPackage shapefile components (.shp, .shx, .dbf, .prj) into individual ZIP archives for upload.\n\n```bash\ngeeup zipshape --input \"./vectors\" --output \"./zipped_vectors\"\n```\n\n**Arguments:**\n- `--input`: Path to directory containing raw shapefiles (required)\n- `--output`: Destination folder for ZIP archives (required)\n\n### Generate Metadata\n\nCreate a metadata CSV file for rasters, defining properties like image IDs, dimensions, and data types.\n\n```bash\ngeeup getmeta --input \"./rasters\" --metadata \"./metadata.csv\"\n```\n\n**Arguments:**\n- `--input`: Path to directory containing raster (TIF) files (required)\n- `--metadata`: Full path for exported metadata CSV file (required)\n\n**Generated Fields:**\n- `system:index`: Asset ID (filename without extension)\n- `xsize`, `ysize`: Raster dimensions\n- `num_bands`: Number of bands\n- `data_type`: Data type (e.g., Byte, Float32)\n- `color_interpretation`: Color interpretation (if not \"Undefined\")\n- `inferred_kind`: Inferred semantic type (image, categorical, continuous)\n\n## Batch Upload\n\n### Upload Images\n\nBatch upload GeoTIFF files to an Earth Engine Image Collection with advanced features like retry logic and parallel processing.\n\n**Basic Usage:**\n```bash\ngeeup upload \\\n  --source \"./rasters\" \\\n  --dest \"users/username/collection\" \\\n  --metadata \"./metadata.csv\" \\\n  --user \"email@gmail.com\"\n```\n\n**Advanced Usage with Parallel Workers:**\n```bash\ngeeup upload \\\n  --source \"./rasters\" \\\n  --dest \"users/username/collection\" \\\n  --metadata \"./metadata.csv\" \\\n  --user \"email@gmail.com\" \\\n  --workers 4 \\\n  --resume \\\n  --pyramids MEAN \\\n  --nodata -9999\n```\n\n**Arguments:**\n\n| Argument | Type | Description |\n|----------|------|-------------|\n| `--source` | Required | Path to directory with images for upload |\n| `--dest` | Required | Destination path for GEE Image Collection (e.g., `users/you/collection`) |\n| `--metadata`, `-m` | Required | Path to metadata CSV file |\n| `--user`, `-u` | Required | Google account email address |\n| `--nodata` | Optional | Integer value to burn as NoData (missing data) |\n| `--mask` | Optional | Boolean (True/False). Use last band as validity mask |\n| `--pyramids` | Optional | Pyramiding policy: MEAN, MODE, MIN, MAX, SAMPLE (default: MEAN) |\n| `--overwrite` | Optional | Overwrite existing assets (pass 'yes' or 'y') |\n| `--dry-run` | Optional | Run validation without uploading |\n| `--workers` | Optional | Number of parallel upload workers (default: 1) |\n| `--max-inflight-tasks` | Optional | Maximum concurrent EE tasks (default: 2800) |\n| `--resume` | Optional | Resume from previous state |\n| `--retry-failed` | Optional | Retry only failed uploads |\n\n**Features:**\n- **Path Normalization**: Automatic handling of legacy (`users/`) and cloud project paths\n- **Collection Auto-Creation**: Prompts to create missing collections and parent folders\n- **State Persistence**: Saves progress to `.geeup-state.json` for resumption\n- **Parallel Uploads**: Multi-threaded GCS uploads with progress bars\n- **Task Throttling**: Automatically waits when task limit is reached\n\n### Upload Tables (Tabup)\n\nBatch upload table data (vectors) from zipped shapefiles or CSVs to a GEE folder.\n\n**Basic Usage:**\n```bash\ngeeup tabup \\\n  --source \"./zipped_vectors\" \\\n  --dest \"users/username/folder\" \\\n  --user \"email@gmail.com\"\n```\n\n**CSV with Geometry Columns:**\n```bash\ngeeup tabup \\\n  --source \"./csv_files\" \\\n  --dest \"users/username/folder\" \\\n  --user \"email@gmail.com\" \\\n  --x \"longitude\" \\\n  --y \"latitude\"\n```\n\n**Arguments:**\n\n| Argument | Type | Description |\n|----------|------|-------------|\n| `--source` | Required | Path to directory with zipped files or CSV files |\n| `--dest` | Required | Destination path for GEE Folder (e.g., `users/you/folder`) |\n| `--user`, `-u` | Required | Google account email address |\n| `--x` | Optional | Column name containing longitude values (for CSVs) |\n| `--y` | Optional | Column name containing latitude values (for CSVs) |\n| `--metadata` | Optional | Path to CSV with metadata |\n| `--overwrite` | Optional | Overwrite existing assets (pass 'yes' or 'y') |\n| `--dry-run` | Optional | Run validation without uploading |\n| `--workers` | Optional | Number of parallel workers (default: 1) |\n| `--max-inflight-tasks` | Optional | Maximum concurrent EE tasks (default: 2500) |\n| `--resume` | Optional | Resume from previous state |\n| `--retry-failed` | Optional | Retry only failed uploads |\n| `--max-error-meters` | Optional | Maximum geometry error in meters (default: 1.0) |\n| `--max-vertices` | Optional | Maximum vertices per geometry (default: 1000000) |\n\n**Features:**\n- Supports both zipped shapefiles and CSV files\n- Automatic folder creation with confirmation\n- State persistence for resumable uploads\n- Progress tracking with file-level details\n\n## Monitoring\n\n### Check Quota\n\nDisplay Earth Engine storage usage and asset counts with visual progress bars.\n\n**Check all quotas:**\n```bash\ngeeup quota\n```\n\n**Check specific project:**\n```bash\ngeeup quota --project \"projects/my-project\"\n```\n\n**Arguments:**\n- `--project`: Project name in format `projects/project-name/assets/` (optional)\n\n**Output includes:**\n- Storage usage (bytes/GB)\n- Asset count\n- Visual progress bars\n- Support for both legacy and cloud projects\n\n### Task Status\n\nQuery the status of Earth Engine tasks with rich formatting.\n\n**Summary of all tasks:**\n```bash\ngeeup tasks\n```\n\n**Filter by state:**\n```bash\ngeeup tasks --state RUNNING\n```\n\n**Query specific task:**\n```bash\ngeeup tasks --id \"TASK_ID_HERE\"\n```\n\n**Arguments:**\n- `--state`: Filter by state (COMPLETED, READY, RUNNING, FAILED, CANCELLED)\n- `--id`: Query specific task ID\n\n### Cancel Tasks\n\nCancel Earth Engine tasks with progress tracking.\n\n**Cancel all tasks:**\n```bash\ngeeup cancel --tasks all\n```\n\n**Cancel running tasks:**\n```bash\ngeeup cancel --tasks running\n```\n\n**Cancel pending tasks:**\n```bash\ngeeup cancel --tasks pending\n```\n\n**Cancel specific task:**\n```bash\ngeeup cancel --tasks \"TASK_ID_HERE\"\n```\n\n**Arguments:**\n- `--tasks`: Specify 'all', 'running', 'pending', or a specific task ID (required)\n\n## Utilities\n\n### Delete Assets\n\nRecursively delete an Earth Engine asset (collection, folder, or image).\n\n```bash\ngeeup delete --id \"users/username/test_collection\"\n```\n\n**Arguments:**\n- `--id`: Full path to asset for deletion (required)\n\n**Warning:** This action cannot be undone. Use with caution.\n\n### Open Documentation\n\nOpen the official geeup documentation in your default web browser.\n\n```bash\ngeeup readme\n```\n\n## Full Documentation\n\nFind the complete documentation at: [https://geeup.geetools.xyz](https://geeup.geetools.xyz)\n\n## Changelog\n\n### Version 2.0.0\n- Enhanced path normalization for legacy and cloud projects\n- Added service account authentication support\n- Improved parallel processing with worker threads\n- Added resume and retry capabilities for failed uploads\n- Enhanced metadata generation with GDAL integration\n- Better error handling and progress tracking\n- Rich console output with visual progress bars\n- State persistence for resumable workflows\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nThis project is licensed under the Apache License 2.0 - see the LICENSE file for details.\n\n## Acknowledgments\n\nThis tool borrows features from another project of mine, [geeadd](https://geeadd.geetools.xyz), such as quota estimation, task monitoring, and cancellation capabilities.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamapriya%2Fgeeup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsamapriya%2Fgeeup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamapriya%2Fgeeup/lists"}