https://github.com/milesgranger/cramjam
Your go-to for easy access to a plethora of compression algorithms, all neatly bundled in one simple installation.
https://github.com/milesgranger/cramjam
brotli bzip2 compression decompression deflate gzip lz4 python rust snappy zstd
Last synced: 7 months ago
JSON representation
Your go-to for easy access to a plethora of compression algorithms, all neatly bundled in one simple installation.
- Host: GitHub
- URL: https://github.com/milesgranger/cramjam
- Owner: milesgranger
- License: mit
- Created: 2020-03-08T07:46:33.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2024-04-23T07:23:06.000Z (over 1 year ago)
- Last Synced: 2024-04-24T04:47:32.676Z (over 1 year ago)
- Topics: brotli, bzip2, compression, decompression, deflate, gzip, lz4, python, rust, snappy, zstd
- Language: Rust
- Homepage:
- Size: 53.5 MB
- Stars: 78
- Watchers: 4
- Forks: 5
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
# cramjam
[](https://github.com/python/black)
[](https://github.com/milesgranger/cramjam/actions/workflows/CI.yml)
[](https://pypi.org/project/cramjam)
[](https://anaconda.org/conda-forge/cramjam)
[](https://pepy.tech/project/cramjam)
[API Documentation](https://docs.rs/cramjam)
### Install
```commandline
pip install --upgrade cramjam # Requires no Python or system dependencies!
```
### CLI
A CLI interface is available as [`cramjam-cli`](https://github.com/cramjam/cramjam-cli)
### libcramjam
A Rust crate and C friendly library available at [libcramjam](https://github.com/cramjam/libcramjam)
---
Extremely thin and easy-to-install Python bindings to de/compression algorithms in Rust.
Allows for using algorithms such as Snappy, without any system or other python dependencies.
---
##### Benchmarks
Some basic benchmarks are available [in the benchmarks directory](./benchmarks/README.md)
---
Available algorithms:
- [X] Snappy `cramjam.snappy`
- [X] Brotli `cramjam.brotli`
- [X] Bzip2 `cramjam.bzip2`
- [X] Lz4 `cramjam.lz4`
- [X] Gzip `cramjam.gzip`
- [X] Zlib `cramjam.zlib`
- [X] Deflate `cramjam.deflate`
- [X] ZSTD `cramjam.zstd`
- [X] XZ / LZMA `cramjam.xz`
Experimental (Requires build from source enabling each feature):
- [X] Blosc2 `cramjam.experimental.blosc2`
- [X] ISA-L backend _(only on 64-bit targets)_
- [X] igzip `cramjam.experimental.igzip`
- [X] ideflate `cramjam.experimental.ideflate`
- [X] izlib `cramjam.experimental.izlib`
All available for use as:
```python
>>> import cramjam
>>> import numpy as np
>>> compressed = cramjam.snappy.compress(b"bytes here")
>>> decompressed = cramjam.snappy.decompress(compressed)
>>> decompressed
cramjam.Buffer(len=10) # an object which implements the buffer protocol
>>> bytes(decompressed)
b"bytes here"
>>> np.frombuffer(decompressed, dtype=np.uint8)
array([ 98, 121, 116, 101, 115, 32, 104, 101, 114, 101], dtype=uint8)
```
Where the API is `cramjam..compress/decompress` and accepts
`bytes`/`bytearray`/`numpy.array`/`cramjam.File`/`cramjam.Buffer` / `memoryview` objects.
**de/compress_into**
Additionally, all variants support `decompress_into` and `compress_into`.
Ex.
```python
>>> import numpy as np
>>> from cramjam import snappy, Buffer
>>>
>>> data = np.frombuffer(b'some bytes here', dtype=np.uint8)
>>> data
array([115, 111, 109, 101, 32, 98, 121, 116, 101, 115, 32, 104, 101,
114, 101], dtype=uint8)
>>>
>>> compressed = Buffer()
>>> snappy.compress_into(data, compressed)
33 # 33 bytes written to compressed buffer
>>>
>>> compressed.tell() # Where is the buffer position?
33 # goodie!
>>>
>>> compressed.seek(0) # Go back to the start of the buffer so we can prepare to decompress
>>> decompressed = b'0' * len(data) # let's write to `bytes` as output
>>> decompressed
b'000000000000000'
>>>
>>> snappy.decompress_into(compressed, decompressed)
15 # 15 bytes written to decompressed
>>> decompressed
b'some bytes here'
```