Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/ldeakin/zarrs_tools

Various tools for creating and manipulating Zarr v3 data with the zarrs rust crate
https://github.com/ldeakin/zarrs_tools

ome-zarr rust zarr zarr-v3 zarrs

Last synced: 6 days ago
JSON representation

Various tools for creating and manipulating Zarr v3 data with the zarrs rust crate

Host: GitHub
URL: https://github.com/ldeakin/zarrs_tools
Owner: LDeakin
License: apache-2.0
Created: 2023-12-04T23:11:23.000Z (11 months ago)
Default Branch: main
Last Pushed: 2024-11-01T03:52:16.000Z (6 days ago)
Last Synced: 2024-11-01T04:28:29.826Z (6 days ago)
Topics: ome-zarr, rust, zarr, zarr-v3, zarrs
Language: Rust
Homepage:
Size: 713 KB
Stars: 15
Watchers: 2
Forks: 3
Open Issues: 2
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md

Awesome Lists containing this project

README

        # zarrs_tools 

[![Latest Version](https://img.shields.io/crates/v/zarrs_tools.svg)](https://crates.io/crates/zarrs_tools)

![msrv](https://img.shields.io/crates/msrv/zarrs_tools)

[![build](https://github.com/LDeakin/zarrs_tools/actions/workflows/ci.yml/badge.svg)](https://github.com/LDeakin/zarrs_tools/actions/workflows/ci.yml)

Various tools for creating and manipulating [Zarr v3](https://zarr.dev) data with the [zarrs](https://github.com/LDeakin/zarrs) rust crate.

A changelog can be found [here](https://github.com/LDeakin/zarrs_tools/blob/main/CHANGELOG.md).

## Tools

All tools support input and output of [Zarr V3](https://zarr-specs.readthedocs.io/en/latest/v3/core/v3.0.html) data.

Some tools additionally support input of a [V3 compatible subset](https://docs.rs/zarrs/latest/zarrs/#implementation-status) of [Zarr V2](https://zarr-specs.readthedocs.io/en/latest/v2/v2.0.html).

- [zarrs_reencode](https://github.com/LDeakin/zarrs_tools/blob/main/docs/zarrs_reencode.md): reencode an array. Manipulate the chunk size, shard size, codecs, fill value, chunk key encoding separator, and attributes.

- [zarrs_filter](https://github.com/LDeakin/zarrs_tools/blob/main/docs/zarrs_filter.md) (feature `filter`): apply simple image filters (transformations) to an array.

- [zarrs_ome](https://github.com/LDeakin/zarrs_tools/blob/main/docs/zarrs_ome.md) (feature `ome`): convert an array to an [OME-Zarr](https://ngff.openmicroscopy.org/latest/index.html) multi-scale image.

  - Supports OME-Zarr `0.5-dev` (as Zarr V3) and `0.5-dev1`. The first is recognised by [Neuroglancer](https://github.com/google/neuroglancer).

- zarrs_info (feature `info`): return metadata related info or the range/histogram of an array.

- [zarrs_binary2zarr](https://github.com/LDeakin/zarrs_tools/blob/main/docs/zarrs_binary2zarr.md) (feature `binary2zarr`): create an array from piped binary data.

- [zarrs_ncvar2zarr](https://github.com/LDeakin/zarrs_tools/blob/main/docs/zarrs_ncvar2zarr.md) (feature `ncvar2zarr`): convert a netCDF variable to an array.

See [docs/](https://github.com/LDeakin/zarrs_tools/blob/main/docs/) for tool documentation.

## `zarrs` Benchmarking

- zarrs_reencode: suitable for round trip benchmarking.

- zarrs_benchmark_read_sync (feature `benchmark`): benchmark the zarrs sync API.

- zarrs_benchmark_read_async (feature `benchmark`): benchmark the zarrs async API.

See the [LDeakin/zarr_benchmarks](https://github.com/LDeakin/zarr_benchmarks) repository for benchmarks of `zarrs` against other Zarr V3 implementations.

## Install

### From [crates.io](https://crates.io/crates/zarrs_tools)

```bash

cargo install --all-features zarrs_tools

```

### From [source](https://github.com/LDeakin/zarrs_tools)

```bash

cargo install --all-features --path .

# cargo install --all-features --git https://github.com/LDeakin/zarrs_tools

```

### Enabling SIMD intrinsics

Encoding and decoding performance may be improved with `avx2`/`sse2` enabled (if supported).

This can be enabled by compiling with either of:

 - `RUSTFLAGS="-C target-cpu=native"`

 - `RUSTFLAGS="-C target-feature=+avx2,+sse2"`

### Enabling non-default `zarrs` codecs

Non-default `zarrs` codecs (see [`zarrs` crate features](https://docs.rs/zarrs/latest/zarrs/#crate-features)) can be enabled by passing them as feature flags.

For example:

```bash

cargo install zarrs_tools --all-features --features zarrs/bitround,zarrs/zfp,zarrs/bz2,zarrs/pcodec

```

## Licence

`zarrs_tools` is licensed under either of

 - the Apache License, Version 2.0 [LICENSE-APACHE](./LICENCE-APACHE) or  or

 - the MIT license [LICENSE-MIT](./LICENCE-MIT) or , at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.