{"id":13735174,"url":"https://github.com/stac-utils/stac-task","last_synced_at":"2025-09-07T05:34:55.393Z","repository":{"id":56712317,"uuid":"523929556","full_name":"stac-utils/stac-task","owner":"stac-utils","description":"Provides a class interface for running custom algorithms on STAC ItemCollections","archived":false,"fork":false,"pushed_at":"2024-12-23T21:00:51.000Z","size":1006,"stargazers_count":22,"open_issues_count":18,"forks_count":5,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-08-26T18:51:42.618Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stac-utils.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-08-12T02:20:38.000Z","updated_at":"2024-12-06T20:07:28.000Z","dependencies_parsed_at":"2023-11-27T15:28:23.644Z","dependency_job_id":"b270be87-46b4-4eea-bd65-d1e9c67ae421","html_url":"https://github.com/stac-utils/stac-task","commit_stats":{"total_commits":170,"total_committers":9,"mean_commits":18.88888888888889,"dds":"0.42352941176470593","last_synced_commit":"320bf30c1375156e5e84232e80a3090fd3f5664e"},"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/stac-utils/stac-task","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stac-utils%2Fstac-task","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stac-utils%2Fstac-task/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stac-utils%2Fstac-task/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stac-utils%2Fstac-task/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stac-utils","download_url":"https://codeload.github.com/stac-utils/stac-task/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stac-utils%2Fstac-task/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273997425,"owners_count":25204538,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-07T02:00:09.463Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T03:01:03.786Z","updated_at":"2025-09-07T05:34:55.369Z","avatar_url":"https://github.com/stac-utils.png","language":"Python","funding_links":[],"categories":["`Python` processing of optical imagery (non deep learning)"],"sub_categories":["Cloud Native Geospatial"],"readme":"\u003c!-- omit from toc --\u003e\n# STAC Task (stac-task)\n\n[![Build Status](https://github.com/stac-utils/stac-task/workflows/CI/badge.svg?branch=main)](https://github.com/stac-utils/stac-task/actions/workflows/continuous-integration.yml)\n[![PyPI version](https://badge.fury.io/py/stac-task.svg)](https://badge.fury.io/py/stac-task)\n[![Documentation Status](https://readthedocs.org/projects/stac-task/badge/?version=latest)](https://stac-task.readthedocs.io/en/latest/?badge=latest)\n[![codecov](https://codecov.io/gh/stac-utils/stac-task/branch/main/graph/badge.svg)](https://codecov.io/gh/stac-utils/stac-task)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\n- [Quickstart for Creating New Tasks](#quickstart-for-creating-new-tasks)\n- [Task Input](#task-input)\n  - [ProcessDefinition Object](#processdefinition-object)\n    - [UploadOptions Object](#uploadoptions-object)\n      - [path\\_template](#path_template)\n      - [collections](#collections)\n    - [tasks](#tasks)\n    - [TaskConfig Object](#taskconfig-object)\n    - [workflow_options](#workflow_options)\n  - [Full ProcessDefinition Example](#full-processdefinition-example)\n- [Migration](#migration)\n  - [0.4.x -\\\u003e 0.5.x](#04x---05x)\n  - [0.5.x -\\\u003e 0.6.0](#05x---060)\n- [Development](#development)\n- [Contributing](#contributing)\n\nThis Python library consists of the Task class, which is used to create custom tasks\nbased on a \"STAC In, STAC Out\" approach. The Task class acts as wrapper around custom\ncode and provides several convenience methods for modifying STAC Items, creating derived\nItems, and providing a CLI.\n\nThis library is based on a [branch of cirrus-lib](https://github.com/cirrus-geo/cirrus-lib/tree/features/task-class)\nexcept aims to be more generic.\n\n## Quickstart for Creating New Tasks\n\n```python\nfrom typing import Any\n\nfrom stactask import Task, DownloadConfig\n\nclass MyTask(Task):\n    name = \"my-task\"\n    description = \"this task does it all\"\n\n    def validate(self, payload: dict[str, Any]) -\u003e bool:\n        return len(self.items) == 1\n\n    def process(self, **kwargs: Any) -\u003e list[dict[str, Any]]:\n        item = self.items[0]\n\n        # download a datafile\n        item = self.download_item_assets(\n            item,\n            config=DownloadConfig(include=['data'])\n        )\n\n        # operate on the local file to create a new asset\n        item = self.upload_item_assets_to_s3(item)\n\n        # this task returns a single item\n        return [item.to_dict(include_self_link=True, transform_hrefs=False)]\n```\n\n## Task Input\n\nTask input is often referred to as a 'payload'.\n\n| Field Name | Type                      | Description                                         |\n| ---------- | ------------------------- | --------------------------------------------------- |\n| type       | string                    | Must be FeatureCollection                           |\n| features   | [Item]                    | An array of STAC Items                              |\n| process    | [`ProcessDefinition`]     | An array of `ProcessDefinition` objects.            |\n| ~~process~~  | ~~`ProcessDefinition`~~ | **DEPRECATED** A `ProcessDefinition` object         |\n\n### ProcessDefinition Object\n\nA Task can be provided additional configuration via the 'process' field in the input\npayload.\n\n| Field Name       | Type               | Description                                                              |\n| ---------------- | ------------------ | ------------------------------------------------------------------------ |\n| description      | string             | Description of the process configuration                                 |\n| upload_options   | `UploadOptions`    | An `UploadOptions` object                                                |\n| tasks            | Map\u003cstr, Map\u003e      | Dictionary of task configurations.                                       |\n| ~~tasks~~        | ~~[`TaskConfig`]~~ | **DEPRECATED** A list of `TaskConfig` objects.                           |\n| workflow_options | Map\u003cstr, Any\u003e      | Dictionary of configuration options applied to all tasks in the workflow |\n\n\n#### UploadOptions Object\n\nOptions used when uploading Item assets to a remote server can be specified in a\n'upload_options' field in the `ProcessDefinition` object.\n\n| Field Name    | Type          | Description                                                                             |\n| ------------- | ------------- | --------------------------------------------------------------------------------------- |\n| path_template | string        | **REQUIRED** A string template for specifying the location of uploaded assets           |\n| public_assets | [str]         | A list of asset keys that should be marked as public when uploaded                      |\n| headers       | Map\u003cstr, str\u003e | A set of key, value headers to send when uploading data to s3                           |\n| collections   | Map\u003cstr, str\u003e | A mapping of output collection name to a JSONPath pattern (for matching Items)          |\n| s3_urls       | bool          | Controls if the final published URLs should be an s3 (s3://*bucket*/*key*) or https URL |\n\n##### path_template\n\nThe 'path_template' string is a way to control the output location of uploaded assets\nfrom a STAC Item using metadata from the Item itself. The template can contain fixed\nstrings along with variables used for substitution. See [the PySTAC documentation for\n`LayoutTemplate`](https://pystac.readthedocs.io/en/stable/api/layout.html#pystac.layout.LayoutTemplate)\nfor a list of supported template variables and their meaning.\n\n##### collections\n\nThe 'collections' dictionary provides a collection ID and JSONPath pattern for matching\nagainst STAC Items. At the end of processing, before the final STAC Items are returned,\nthe Task class can be used to assign all of the Items to specific collection IDs. For\neach Item the JSONPath pattern for all collections will be compared. The first match\nwill cause the Item's Collection ID to be set to the provided value.\n\nFor example:\n\n```json\n\"collections\": {\n    \"landsat-c2l2\": \"$[?(@.id =~ 'LC08.*')]\"\n}\n```\n\nIn this example, the task will set any STAC Items that have an ID beginning with \"LC08\"\nto the `landsat-c2l2` collection.\n\nSee [JSONPath Online Evaluator](https://jsonpath.com) to experiment with JSONPath and\n[regex101](https://regex101.com) to experiment with regex.\n\n#### tasks\n\nThe 'tasks' field is a dictionary with an optional key for each task. If present, it\ncontains a dictionary that is converted to a set of keywords and passed to the Task's\n`process` function. The documentation for each Task will provide the list of available\nparameters.\n\n```json\n{\n    \"tasks\": {\n        \"task-a\": {\n            \"param1\": \"value1\"\n        },\n        \"task-c\": {\n            \"param2\": \"value2\"\n        }\n    }\n}\n```\n\nIn the example above, a task named `task-a` would have the `param1=value1` passed as a\nkeyword, while `task-c` would have `param2=value2` passed. If there were a `task-b` to\nbe run, it would not be passed any keywords.\n\n#### TaskConfig Object\n\n**DEPRECATED** The 'tasks' field _should_ be a dictionary of parameters, with task names\nas keys. See [tasks](#tasks) for more information. `TaskConfig` objects are supported\nfor backwards compatibility.\n\n| Field Name | Type          | Description                                                                         |\n| ---------- | ------------- | ----------------------------------------------------------------------------------- |\n| name       | str           | **REQUIRED** Name of the task                                                       |\n| parameters | Map\u003cstr, str\u003e | Dictionary of keyword parameters that will be passed to the Task `process` function |\n\n#### workflow_options\n\nThe 'workflow_options' field is a dictionary of options that apply to all tasks in the\nworkflow. The 'workflow_options' dictionary is combined with each task's option\ndictionary. If a key in the 'workflow_options' dictionary conflicts with a key in a\ntask's option dictionary, the task option value takes precedence.\n\n### Full ProcessDefinition Example\n\n```json\n{\n    \"description\": \"My process configuration\",\n    \"upload_options\": {\n        \"path_template\": \"s3://my-bucket/${collection}/${year}/${month}/${day}/${id}\",\n        \"collections\": {\n            \"landsat-c2l2\": \"$[?(@.id =~ 'LC08.*')]\"\n        }\n    },\n    \"tasks\": {\n        \"task-name\": {\n            \"param\": \"value\"\n        }\n    }\n}\n```\n\n## Migration\n\n### 0.4.x -\u003e 0.5.x\n\nIn 0.5.0, the previous use of fsspec to download Item Assets has been replaced with the\nstac-asset library. This has necessitated a change in the parameters that the download\nmethods accept.\n\nThe primary change is that the Task methods `download_item_assets` and\n`download_items_assets` (items plural) now accept fewer explicit and implicit (kwargs)\nparameters.\n\nPreviously, the methods looked like:\n\n```python\n  def download_item_assets(\n        self,\n        item: Item,\n        path_template: str = \"${collection}/${id}\",\n        keep_original_filenames: bool = False,\n        **kwargs: Any,\n    ) -\u003e Item:\n```\n\nbut now look like:\n\n```python\n    def download_item_assets(\n        self,\n        item: Item,\n        path_template: str = \"${collection}/${id}\",\n        config: Optional[DownloadConfig] = None,\n    ) -\u003e Item:\n```\n\nSimilarly, the `asset_io` package methods were previously:\n\n```python\nasync def download_item_assets(\n    item: Item,\n    assets: Optional[list[str]] = None,\n    save_item: bool = True,\n    overwrite: bool = False,\n    path_template: str = \"${collection}/${id}\",\n    absolute_path: bool = False,\n    keep_original_filenames: bool = False,\n    **kwargs: Any,\n) -\u003e Item:\n```\n\nand are now:\n\n```python\nasync def download_item_assets(\n    item: Item,\n    path_template: str = \"${collection}/${id}\",\n    config: Optional[DownloadConfig] = None,\n) -\u003e Item:\n```\n\nAdditionally, `kwargs` keys were set to pass configuration through to fsspec. The most\ncommon parameter was `requester_pays`, to set the Requester Pays flag in AWS S3\nrequests.\n\nMany of these parameters can be directly translated into configuration passed in a\n`DownloadConfig` object, which is just a wrapper over the `stac_asset.Config` object.\n\nMigration of these various parameters to `DownloadConfig` are as follows:\n\n- `assets`: set `include`\n- `requester_pays`: set `s3_requester_pays` = True\n- `keep_original_filenames`: set `file_name_strategy` to\n  `FileNameStrategy.FILE_NAME` if True or `FileNameStrategy.KEY` if False\n- `overwrite`: set `overwrite`\n- `save_item`: none, Item is always saved\n- `absolute_path`: none. To create or retrieve the Asset hrefs as absolute paths, use\n  either `Item#make_all_asset_hrefs_absolute()` or `Asset#get_absolute_href()`\n\n### 0.5.x -\u003e 0.6.0\n\nPreviously, the `validate` method was a _classmethod_, validating the payload argument\npassed.  This has now been made an instance method, which validates the `self._payload`\ncopy of the payload, from which the `Task` instance is constructed.  This is\nbehaviorally the same, in that construction will fail if validation fails, but allows\nimplementers to utilize the instance method's convenience functions.\n\nPrevious implementations of `validate` would have been similar to this:\n\n```python\n    @classmethod\n    def validate(payload: dict[str, Any]) -\u003e bool:\n        # Check The Things™\n        return isinstance(payload, dict)\n```\n\nAnd will now need to be updated to this form:\n\n```python\n    def validate(self) -\u003e bool:\n        # Check The Things™\n        return isinstance(self._payload, dict)\n```\n\n## Development\n\nClone, install in editable mode with development and test requirements, and install the\n**pre-commit** hooks:\n\n```shell\ngit clone https://github.com/stac-utils/stac-task\ncd stac-task\npip install -e '.[dev,test]'\npre-commit install\n```\n\nTo run the tests:\n\n```shell\npytest\n```\n\nTo lint all the files:\n\n```shell\npre-commit run --all-files\n```\n\n## Contributing\n\nUse Github [issues](https://github.com/stac-utils/stac-task/issues) and [pull\nrequests](https://github.com/stac-utils/stac-task/pulls).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstac-utils%2Fstac-task","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstac-utils%2Fstac-task","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstac-utils%2Fstac-task/lists"}