https://github.com/nh2/lz4-frame-conduit

Haskell Conduit implementing the official LZ4 frame streaming format
https://github.com/nh2/lz4-frame-conduit

haskell haskell-conduit lz4 lz4-frame

Last synced: about 1 month ago
JSON representation

Haskell Conduit implementing the official LZ4 frame streaming format

• Host: GitHub
• URL: https://github.com/nh2/lz4-frame-conduit
• Owner: nh2
• License: mit
• Created: 2017-03-29T01:32:43.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2023-11-15T15:05:06.000Z (about 1 year ago)
• Last Synced: 2024-04-26T19:21:11.742Z (8 months ago)
• Topics: haskell, haskell-conduit, lz4, lz4-frame
• Language: Haskell
• Homepage:
• Size: 79.1 KB
• Stars: 5
• Watchers: 4
• Forks: 4
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# lz4-frame-conduit

Haskell Conduit implementing the official LZ4 frame streaming format.

## Building

Use `git clone --recursive`, because `lz4/` is a git submodule.

## Rationale and comparison to non-`lz4`-compatible libraries

There exist two `lz4` formats:

1. the **block format**, limited to compressing data < 2 GB

2. the **frame format**, suitable for streaming arbitrarily sized data

  ** This is what the `lz4` command line utility uses

**This library implements the frame format.**

Some existing Haskell libraries implement only the block format, and are thus not suitable to compress data > 2 GB in a way.

(Of course they could chunk the output in some arbitrary way, but that wouldn't be compatible with the `lz4` command line utility.)

The libraries that belong to this category at time of writing are:

* [`lz4`](https://hackage.haskell.org/package/lz4)

* [`lz4-conduit`](https://hackage.haskell.org/package/lz4-conduit)