{"id":34036009,"url":"https://github.com/bcdev/zappend","last_synced_at":"2026-04-09T04:31:40.598Z","repository":{"id":216490754,"uuid":"731624717","full_name":"bcdev/zappend","owner":"bcdev","description":"Robustly creating and updating Zarr data cubes from smaller subsets","archived":false,"fork":false,"pushed_at":"2025-04-15T15:59:01.000Z","size":26901,"stargazers_count":48,"open_issues_count":11,"forks_count":1,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-12-15T12:16:36.067Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://bcdev.github.io/zappend/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bcdev.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-14T13:55:38.000Z","updated_at":"2025-12-05T13:28:13.000Z","dependencies_parsed_at":"2024-01-22T13:16:09.690Z","dependency_job_id":"520567ef-c895-4b4d-bb87-1b37135494c6","html_url":"https://github.com/bcdev/zappend","commit_stats":{"total_commits":178,"total_committers":2,"mean_commits":89.0,"dds":0.005617977528089901,"last_synced_commit":"4acf2de920ef288e658ae2d09e167b09ae7104c6"},"previous_names":["bcdev/zappend"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/bcdev/zappend","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcdev%2Fzappend","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcdev%2Fzappend/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcdev%2Fzappend/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcdev%2Fzappend/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bcdev","download_url":"https://codeload.github.com/bcdev/zappend/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcdev%2Fzappend/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31586403,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-08T14:31:17.711Z","status":"online","status_checked_at":"2026-04-09T02:00:06.848Z","response_time":112,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-13T20:08:03.872Z","updated_at":"2026-04-09T04:31:40.592Z","avatar_url":"https://github.com/bcdev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n    \u003cimg src=\"docs/assets/logo.png\" width=\"64\" title=\"zappend\"\u003e\n    \u0026nbsp;zappend\n\u003c/h1\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n[![CI](https://github.com/bcdev/zappend/actions/workflows/tests.yml/badge.svg)](https://github.com/bcdev/zappend/actions/workflows/tests.yml)\n[![codecov](https://codecov.io/gh/bcdev/zappend/graph/badge.svg?token=B3R6bNmAUp)](https://codecov.io/gh/bcdev/zappend)\n[![PyPI Version](https://img.shields.io/pypi/v/zappend)](https://pypi.org/project/zappend/)\n[![Conda Version](https://anaconda.org/conda-forge/zappend/badges/version.svg)](https://anaconda.org/conda-forge/zappend)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v0.json)](https://github.com/charliermarsh/ruff)\n[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/bcdev/zappend/HEAD?labpath=examples%2Fzappend-demo.ipynb)\n[![GitHub License](https://img.shields.io/github/license/bcdev/zappend)](https://github.com/bcdev/zappend)\n\n\u003c/div\u003e\n\n\u003c!--- Align following sections with docs/index.md --\u003e\n\n---\n\n`zappend` is a tool written in Python that is used for robustly creating and \nupdating Zarr datacubes from smaller dataset slices. It is built on top of the \nawesome Python packages [xarray](https://docs.xarray.dev/) and [zarr](https://zarr.readthedocs.io/).\n\n## Motivation\n\nThe objective of `zappend` is enabling geodata scientists and developers to \nrobustly create large data cubes. The tool performs transaction-based dataset \nappends to existing data cubes in the \n[Zarr format](https://zarr.readthedocs.io/en/stable/spec/v2.html). If an error \noccurs during an append step — typically due to I/O problems or out-of-memory \nconditions — `zappend` will automatically roll back the operation, ensuring that \nthe existing data cube maintains its structural integrity. The design drivers \nbehind zappend are first ease of use and secondly, high configurability \nregarding filesystems, data source types, data cube outline and encoding. \n\nThe tool comprises a command-line interface, a Python API for programmatic \ncontrol, and a comprehensible documentation to guide users effectively. \nYou can easily install `zappend` as a plain Python package using either \n`pip install zappend` or `conda install -conda-forge zappend`.\n\n## Features\n\nThe `zappend` tool provides the following features:\n\n* **Locking**: While the target dataset is being modified, a file lock is \n  created, effectively preventing concurrent dataset modifications.\n* **Transaction-based dataset appends**: On failure during an append step, \n  the transaction is rolled back, so that the target dataset remains valid and \n  preserves its integrity.\n* **Filesystem transparency**: The target dataset may be generated and updated \n  in any writable filesystems supported by the \n  [fsspec](https://filesystem-spec.readthedocs.io/) package. \n  The same holds for the slice datasets to be appended.\n* **Dataset polling**: The tool can be configured to wait for slice datasets to \n  become available. \n* **Dynamic attributes**: Use syntax `{{ expression }}` to update the target \n  dataset with dynamically computed attribute values. \n* **CLI and Python API**: The tool can be used in a shell using the \n  [`zappend`](cli.md) command or from Python. When used from Python using the \n  [`zappend()`](api.md) function, slice datasets can be passed as local file \n  paths, URIs, as datasets of type \n  [xarray.Dataset](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html), or as custom\n  [slice sources](https://bcdev.github.io/zappend/guide/#slice-sources).\n\n  \nMore about zappend can be found in its \n[documentation](https://bcdev.github.io/zappend/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbcdev%2Fzappend","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbcdev%2Fzappend","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbcdev%2Fzappend/lists"}