awesome-zarr

🎀 Awesome Zarr resources
https://github.com/dahnj/awesome-zarr

Last synced: 7 days ago
JSON representation

Articles
- Kerchunk
Existing resources
- Existing lists
- - Zarr website
GeoZarr
- N5
Introductory videos
- Existing lists
Libraries
- Existing lists
  - Zarr Implementations
  - xpublish
  - xpublish-community - opendap](https://github.com/xpublish-community/xpublish-opendap)
  - Improving Access to NOAA NOS Model Data with Kerchunk and Xpublish
  - ndpyramid
  - Tensorstore - tensorstore](https://github.com/google/xarray-tensorstore/): library for efficiently reading and writing large multi-dimensional arrays, has Zarr API
  - KivkIO - storage/api-reference-guide/index.html), enabling [GPUDirect Storage](https://developer.nvidia.com/blog/gpudirect-storage/)
  - rechunker - to-disk transformation for chunked arrays
  - xpartition
  - Zarr Encoding Specification
  - Pangeo-forge - source data platform for transforming datasets into [analysis-ready cloud-optimized](https://ieeexplore.ieee.org/document/9354557) formats.
  - Pangeo Forge in 4 minutes - Ryan Abernathey | SciPy 2022](https://www.youtube.com/watch?v=sY20UpYCAEE&ab_channel=Enthought)
  - numcodecs
  - traverzarr
  - zarr_checksum
  - zarrdump
  - Improving Access to NOAA NOS Model Data with Kerchunk and Xpublish
  - Pangeo Forge in 4 minutes - Ryan Abernathey | SciPy 2022](https://www.youtube.com/watch?v=sY20UpYCAEE&ab_channel=Enthought)
  - pydantic-zarr
- Kerchunk
Life sciences
- Kerchunk
  - bdz - based format for storing quantitative biosystems dynamics data
  - ome-zarr-py - generation file format ([NGFF](https://ngff.openmicroscopy.org/)) specifications for storing bioimaging data in the cloud.
  - ez_zarr - level access to OME-Zarr filesets
  - hdmf-zarr
  - Zarr | Life Science Lightning Talk | Trevor Manz | Dask Summit 2021
  - Accelerating Single-cell Bioinformatics with N-dimensional Arrays in the Cloud | ISMMS
  - What are next-generation file formats (NGFF)?
  - Zarr | Life Science Lightning Talk | Trevor Manz | Dask Summit 2021
- Visualization
  - Image viewers with OME-Zarr support
  - WEBKNOSSOS - based visualization & annotation tool, supports OME-Zarr
  - Napari
  - Vizarr - dbmi/viv) (OME-Zarr and OME-TIFF)
  - Neuroglancer - based viewer for volumetric data
  - BigDataViewer
Talks & Videos
- Kerchunk
  - Zarr: Introductory Talks
  - Zarr Developers playlists
  - Zarr: Zarr: Projects, Uses, Research and Workflows
  - Zarr Talks
  - Earthmover Webinar: Building a Planetary Scale Earth Observation Data Cube in Zarr - mover/serverless-datacube-demo) and [slides](https://app.hubspot.com/documents/22822350/view/788058450?accessId=9423df)
  - - mover/forecast-datacube-demo) and [slides](https://app.hubspot.com/documents/22822350/view/814797193?accessId=e3acdd)
  - Presentation | Zarr: Community specification of large, cloud-optimised, N-dimensional, typed array storage
  - SciPy 2023
  - - mover/forecast-datacube-demo) and [slides](https://app.hubspot.com/documents/22822350/view/814797193?accessId=e3acdd)
  - Zarr Developers playlists
Zarr & other array data formats
- COG: Cloud-Optimized GeoTIFF
- N5
  - 1 - specs.readthedocs.io/en/latest/v3/core/v3.0.html))
  - n5
  - zarr.n5
  - z5
  - Zarr N5 spec diff (zarr-specs#3)
- NetCDF & HDF5
  - xarray - started-guide/why-xarray.html#core-data-structures)
  - use HDF5 as a backend
  - NCZarr
  - A Comparison of HDF5, Zarr, and netCDF4 in Performing Common I/O Operations
  - Pangeo: HDF5 at the speed of Zarr
  - Joe Jevnik: Zarr vs. HDF5 | PyData New York 2019
  - Pangeo: HDF5 at the speed of Zarr
  - Joe Jevnik: Zarr vs. HDF5 | PyData New York 2019
- Visualization
Zarr & STAC
- N5
Zarr V3
- Existing lists

Programming Languages

Python 8 Jupyter Notebook 3 C++ 2 TypeScript 2 Java 1

Categories

Libraries 29 Zarr & other array data formats 20 Life sciences 14 Zarr & STAC 12 Talks & Videos 10 Zarr V3 10 GeoZarr 6 Introductory videos 4 Existing resources 4 Articles 3

Sub Categories

Existing lists 36 Kerchunk 31 N5 23 Visualization 9 NetCDF & HDF5 8 COG: Cloud-Optimized GeoTIFF 4

Keywords

zarr 3 ngff 1 ome 1 ome-zarr 1 chunked-storage 1 cloud 1 h5py 1 multidimensional-arrays 1 multidimensional-data 1 n5 1 storage 1 gehlenborglab 1 imjoy 1 jupyter-notebooks 1 viv 1 fast-api 1 rest-api 1 xarray 1