Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/cgohlke/imagecodecs

Image transformation, compression, and decompression codecs
https://github.com/cgohlke/imagecodecs

avif image jpeg jpeg-ls jpeg-xl jpeg-xr jpeg-xs jpeg2000 lerc lz4 lzf lzma lzw packbits python tiff ultrahdr webp zarr zfp

Last synced: 7 days ago
JSON representation

Image transformation, compression, and decompression codecs

Awesome Lists containing this project

README 

        
..

  This file is generated by setup.py



Image transformation, compression, and decompression codecs

===========================================================



Imagecodecs is a Python library that provides block-oriented, in-memory buffer

transformation, compression, and decompression functions for use in Tifffile,

Czifile, Zarr, kerchunk, and other scientific image input/output packages.



Decode and/or encode functions are implemented for Zlib (DEFLATE), GZIP, LZMA,

ZStandard (ZSTD), Blosc, Brotli, Snappy, BZ2, LZ4, LZ4F, LZ4HC, LZ4H5, LZW,

LZO, LZF, LZFSE, LZHAM, PGLZ (PostgreSQL LZ), RCOMP (Rice), ZFP, SZ3, Pcodec,

SPERR, AEC, SZIP, LERC, EER, NPY, BCn, DDS, BMP, PNG, APNG, GIF, TIFF, WebP,

JPEG 8 and 12-bit, Lossless JPEG (LJPEG, LJ92, JPEGLL), JPEG 2000 (JP2, J2K),

JPEG LS, JPEG XL, JPEG XS, JPEG XR (WDP, HD Photo), Ultra HDR (JPEG_R),

MOZJPEG, AVIF, HEIF, QOI, RGBE (HDR), Jetraw, DICOMRLE, PackBits,

Packed Integers, Delta, XOR Delta, Floating Point Predictor, Bitorder reversal,

Byteshuffle, Bitshuffle, Float24 (24-bit floating point),

Quantize (Scale, BitGroom, BitRound, GranularBR), and

CMS (color space transformations).

Checksum functions are implemented for crc32, adler32, fletcher32, and

Jenkins lookup3.



:Author: `Christoph Gohlke `_

:License: BSD 3-Clause

:Version: 2024.9.22

:DOI: `10.5281/zenodo.6915978 `_



Quickstart

----------



Install the imagecodecs package and all dependencies from the

`Python Package Index `_::



    python -m pip install -U "imagecodecs[all]"



Imagecodecs is also available in other package repositories such as

`Anaconda `_,

`MSYS2 `_, and

`MacPorts `_.



See `Requirements`_ and `Notes`_ for building from source.



See `Examples`_ for using the programming interface.



Source code and support are available on

`GitHub `_.



Requirements

------------



This revision was tested with the following requirements and dependencies

(other versions may work):



- `CPython `_ 3.10.11, 3.11.9, 3.12.6, 3.13.0rc2 64-bit

- `Numpy `_ 2.1.1

- `numcodecs `_ 0.13.0

  (optional, for Zarr compatible codecs)



Build requirements:



- `Cython `_ 3.0.11

- `brotli `_ 1.1.0

- `brunsli `_ 0.1

- `bzip2 `_ 1.0.8

- `c-blosc `_ 1.21.6

- `c-blosc2 `_ 2.15.1

- `charls `_ 2.4.2

- `giflib `_ 5.2.2

- `jetraw `_ 23.03.16.4

- `jxrlib `_ 1.2

- `lcms2 `_ 2.16

- `lerc `_ 4.0.4

- `libaec `_ 1.1.3

- `libavif `_ 1.1.1

  (`aom `_ 3.10.0,

  `dav1d `_ 1.4.3,

  `rav1e `_ 0.7.1,

  `svt-av1 `_ 2.2.1

  `libyuv `_ main)

- `libdeflate `_ 1.21

- `libheif `_ 1.18.2

  (`libde265 `_ 1.0.15,

  `x265 `_ 3.6)

- `libjpeg-turbo `_ 3.0.4

- `libjxl `_ 0.11.0

- `libjxs `_ 2.0.2

- `liblzma `_ 5.6.2

- `libpng `_ 1.6.44

- `libpng-apng `_ 1.6.44

- `libtiff `_ 4.7.0

- `libultrahdr `_ 1.2.0

- `libwebp `_ 1.4.0

- `lz4 `_ 1.10.0

- `lzfse `_ 1.0

- `lzham_codec `_ 1.0

- `lzokay `_ db2df1f

- `mozjpeg `_ 4.1.5

- `openjpeg `_ 2.5.2

- `pcodec `_ 0.3.1

- `snappy `_ 1.2.1

- `sperr `_ 0.8.2

- `sz3 `_ 3.1.8 (3.2.0 crashes)

- `zfp `_ 1.0.1

- `zlib `_ 1.3.1

- `zlib-ng `_ 2.2.2

- `zopfli `_ 1.0.3

- `zstd `_ 1.5.6



Vendored requirements:



- `bcdec.h `_ 3b29f8f

- `bitshuffle `_ 0.5.1

- `cfitsio ricecomp.c `_ modified

- `h5checksum.c `_ modified

- `jpg_0XC3.cpp

  `_

  modified

- `liblj92

  `_ modified

- `liblzf `_ 3.6

- `libspng `_ 0.7.4

- `nc4var.c `_

  modified

- `pg_lzcompress.c `_ modified

- `qoi.h `_ 36190eb

- `rgbe.c `_ modified



Test requirements:



- `tifffile `_ 2024.9.20

- `czifile `_ 2019.7.2

- `zarr `_ 2.18.2

- `python-blosc `_ 1.11.2

- `python-blosc2 `_ 2.7.1

- `python-brotli `_ 1.0.9

- `python-lz4 `_ 4.3.3

- `python-lzf `_ 0.2.6

- `python-snappy `_ 0.7.2

- `python-zstd `_ 1.5.5.1

- `pyliblzfse `_ 0.4.1

- `zopflipy `_ 1.10



Revisions

---------



2024.9.22



- Pass 7644 tests.

- Use libjpeg-turbo for all Lossless JPEG bit-depths if possible (#105).

- Fix PackBits encoder fails to skip short replication blocks (#107).

- Fix JPEG2K encoder leaving trailing random bytes (#104).

- Fix encoding and decoding JPEG XL with custom bitspersample (#102).

- Improve error handling in lzf_decode (#103).

- Add Ultra HDR (JPEG_R) codec based on libultrahdr library (#108).

- Add JPEGXS codec based on libjxs library (source only).

- Add SZ3 codec based on SZ3 library.

- Deprecate Python 3.9, support Python 3.13.



2024.6.1



- Fix segfault in sperr_decode.

- Fix segfault when strided-decoding into buffers with unexpected shapes (#98).

- Fix jpeg2k_encoder output buffer too small (#101).

- Add PCODEC codec based on pcodec library.

- Support NumPy 2.



2024.1.1



- Add 8/24-bit BMP codec.

- Add SPERR codec based on SPERR library.

- Add LZO decoder based on lzokay library.

- Add DICOMRLE decoder.

- Enable float16 in CMS codec.

- Enable MCT for lossless JPEG2K encoder (#88).

- Ignore pad-byte in PackBits decoder (#86).

- Fix heif_write_callback error message not set.

- Require lcms2 2.16 with issue-420 fixes.

- Require libjxl 0.9, libaec 1.1, Cython 3.



2023.9.18



- Rebuild with updated dependencies fixes CVE-2024-4863.



2023.9.4



- Map avif_encode level parameter to quality (breaking).

- Support monochrome images in avif_encode.

- Add numthreads parameter to avif_decode (fix imread of AVIF).

- Add quantize filter (BitGroom, BitRound, GBR) via nc4var.c.

- Add LZ4H5 codec.

- Support more BCn compressed DDS fourcc types.

- Require libavif 1.0.



2023.8.12



- Add EER (Electron Event Representation) decoder.

- Add option to pass initial value to crc32 and adler32 checksum functions.

- Add fletcher32 and lookup3 checksum functions via HDF5's h5checksum.c.

- Add Checksum codec for numcodecs.



2023.7.10



- Rebuild with optimized compile flags.



2023.7.4



- Add BCn and DDS decoder via bcdec library.

- Add functions to transcode JPEG XL to/from JPEG (#78).

- Add option to decode select frames from animated WebP.

- Use legacy JPEG8 codec when building without libjpeg-turbo 3 (#65).

- Change blosc2_encode defaults to match blosc2-python (breaking).

- Fix segfault writing JPEG2K with more than 4 samples.

- Fix some codecs returning bytearray by default.

- Fully vendor cfitsio's ricecomp.c.

- Drop support for Python 3.8 and numpy < 1.21 (NEP29).



- …



Refer to the CHANGES file for older revisions.



Objectives

----------



Many scientific image storage formats like TIFF, CZI, DICOM, HDF, and Zarr

are containers that hold large numbers of small data segments (chunks, tiles,

stripes), which are encoded using a variety of compression and pre-filtering

methods. Metadata common to all data segments are typically stored separate

from the segments.



The purpose of the Imagecodecs library is to support Python modules in

encoding and decoding such data segments. The specific aims are:



- Provide functions for encoding and decoding small image data segments

  in-memory (not in-file) from and to bytes or numpy arrays for many

  compression and filtering methods.

- Support image formats and compression methods not available elsewhere in

  the Python ecosystem.

- Reduce the runtime dependency on numerous, large, inapt, or unmaintained

  Python packages. The imagecodecs package only depends on numpy.

- Implement codecs as Cython wrappers of 3rd party libraries with a C API

  and permissive license if exists, else use own C library.

  Provide Cython definition files for the wrapped C libraries.

- Release the Python global interpreter lock (GIL) during extended native/C

  function calls for multi-threaded use.



Accessing parts of large data segments and reading metadata from segments

are out of the scope of this library.



Notes

-----



This library is largely a work in progress.



The API is not stable yet and might change between revisions.



Python <= 3.8 is no longer supported. 32-bit versions are deprecated.



Works on little-endian platforms only.



Supported platforms are ``win_amd64``, ``win_arm64``, ``win32``,

``macosx_x86_64``, ``macosx_arm64``, ``manylinux_x86_64``, and

``manylinux_aarch64``.



Wheels may not be available for all platforms and all releases.



Only the ``win_amd64`` wheels include all features.



The ``tiff``, ``bcn``, ``dds``, ``dicomrle``, ``eer``, ``lzo``, ``packints``,

and ``jpegsof3`` codecs are currently decode-only.



The ``heif``, ``jetraw``, and ``jpegxs`` codecs are distributed as source

code only due to license and possible patent usage issues.



The latest `Microsoft Visual C++ Redistributable for Visual Studio 2015-2022

`_

is required on Windows.



Refer to the imagecodecs/licenses folder for 3rd-party library licenses.



This software is based in part on the work of the Independent JPEG Group.



Update pip and setuptools to the latest version before installing imagecodecs::



    python -m pip install -U pip setuptools wheel Cython



Before building imagecodecs from source code, install required tools and

libraries. For example, on latest Ubuntu Linux distributions:



    ``sudo apt-get install build-essential python3-dev cython3 python3-pip

    python3-setuptools python3-wheel python3-numpy libdeflate-dev libjpeg-dev

    libjxr-dev liblcms2-dev liblz4-dev liblerc-dev liblzma-dev libopenjp2-7-dev

    libpng-dev libtiff-dev libwebp-dev libz-dev libzstd-dev``



To build and install imagecodecs from source code, run::



    python -m pip install .



Many extensions are disabled by default when building from source.



To define which extensions are built, or to modify build settings such as

library names and compiler arguments, provide a

``imagecodecs_distributor_setup.customize_build`` function, which is

imported and executed during setup.

See ``setup.py`` for pre-defined ``customize_build`` functions.



Other projects providing imaging or compression codecs:

`Python zlib `_,

`Python bz2 `_,

`Python lzma `_,

`backports.lzma `_,

`python-lzo `_,

`python-lzw `_,

`python-lerc `_,

`wavpack-numcodecs

`_,

`packbits `_,

`isa-l.igzip `_,

`fpzip `_,

`libmng `_,

`OpenEXR `_

(EXR, PIZ, PXR24, B44, DWA),

`pyJetraw `_,

`tinyexr `_,

`pytinyexr `_,

`pyroexr `_,

`JasPer `_,

`libjpeg `_ (GPL),

`pylibjpeg `_,

`pylibjpeg-libjpeg `_ (GPL),

`pylibjpeg-openjpeg `_,

`pylibjpeg-rle `_,

`glymur `_,

`pyheif `_,

`pyrus-cramjam `_,

`PyLZHAM `_,

`BriefLZ `_,

`QuickLZ `_ (GPL),

`LZO `_ (GPL),

`nvJPEG `_,

`nvJPEG2K `_,

`PyTurboJPEG `_,

`CCSDS123 `_,

`LPC-Rice `_,

`CompressionAlgorithms `_,

`Compressonator `_,

`Wuffs `_,

`TinyDNG `_,

`OpenJPH `_,

`Grok `_ (AGPL),

`MAFISC

`_,

`B3D `_,

`fo-dicom.Codecs `_,

`jpegli `_.



Examples

--------



Import the JPEG2K codec:



.. code-block:: python



    >>> from imagecodecs import (

    ...     jpeg2k_encode,

    ...     jpeg2k_decode,

    ...     jpeg2k_check,

    ...     jpeg2k_version,

    ...     JPEG2K,

    ... )



Check that the JPEG2K codec is available in the imagecodecs build:



.. code-block:: python



    >>> JPEG2K.available

    True



Print the version of the JPEG2K codec's underlying OpenJPEG library:



.. code-block:: python



    >>> jpeg2k_version()

    'openjpeg 2.5.2'



Encode a numpy array in lossless JP2 format:



.. code-block:: python



    >>> array = numpy.random.randint(100, 200, (256, 256, 3), numpy.uint8)

    >>> encoded = jpeg2k_encode(array, level=0)

    >>> bytes(encoded[:12])

    b'\x00\x00\x00\x0cjP  \r\n\x87\n'



Check that the encoded bytes likely contain a JPEG 2000 stream:



.. code-block:: python



    >>> jpeg2k_check(encoded)

    True



Decode the JP2 encoded bytes to a numpy array:



.. code-block:: python



    >>> decoded = jpeg2k_decode(encoded)

    >>> numpy.array_equal(decoded, array)

    True



Decode the JP2 encoded bytes to an existing numpy array:



.. code-block:: python



    >>> out = numpy.empty_like(array)

    >>> _ = jpeg2k_decode(encoded, out=out)

    >>> numpy.array_equal(out, array)

    True



Not all codecs are fully implemented, raising exceptions at runtime:



.. code-block:: python



    >>> from imagecodecs import tiff_encode

    >>> tiff_encode(array)

    Traceback (most recent call last):

     ...

    NotImplementedError: tiff_encode



Write the numpy array to a JP2 file:



.. code-block:: python



    >>> from imagecodecs import imwrite, imread

    >>> imwrite('_test.jp2', array)



Read the image from the JP2 file as numpy array:



.. code-block:: python



    >>> image = imread('_test.jp2')

    >>> numpy.array_equal(image, array)

    True



Create a JPEG 2000 compressed Zarr array:



.. code-block:: python



    >>> import zarr

    >>> import numcodecs

    >>> from imagecodecs.numcodecs import Jpeg2k

    >>> numcodecs.register_codec(Jpeg2k)

    >>> zarr.zeros(

    ...     (4, 5, 512, 512, 3),

    ...     chunks=(1, 1, 256, 256, 3),

    ...     dtype='u1',

    ...     compressor=Jpeg2k(),

    ... )

    



Access image data in a sequence of JP2 files via tifffile.FileSequence and

dask.array:



.. code-block:: python



    >>> import tifffile

    >>> import dask.array

    >>> def jp2_read(filename):

    ...     with open(filename, 'rb') as fh:

    ...         data = fh.read()

    ...     return jpeg2k_decode(data)

    ...

    >>> with tifffile.FileSequence(jp2_read, '*.jp2') as ims:

    ...     with ims.aszarr() as store:

    ...         dask.array.from_zarr(store)

    ...

    dask.array>> store.write_fsspec(

    ...     'temp.json', url='file://', codec_id='imagecodecs_jpeg2k'

    ... )

    >>> import fsspec

    >>> mapper = fsspec.get_mapper(

    ...     'reference://', fo='temp.json', target_protocol='file'

    ... )

    >>> zarr.open(mapper, mode='r')

    



View the image in the JP2 file from the command line::



    python -m imagecodecs _test.jp2