https://github.com/iiseymour/pystreamvbyte
Python bindings for streamvbyte
https://github.com/iiseymour/pystreamvbyte
compression integer-compression python simd varint
Last synced: 9 days ago
JSON representation
Python bindings for streamvbyte
- Host: GitHub
- URL: https://github.com/iiseymour/pystreamvbyte
- Owner: iiSeymour
- License: apache-2.0
- Created: 2019-05-05T21:10:33.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2021-07-04T21:11:38.000Z (over 4 years ago)
- Last Synced: 2025-09-19T05:52:46.731Z (5 months ago)
- Topics: compression, integer-compression, python, simd, varint
- Language: Python
- Homepage:
- Size: 17.6 KB
- Stars: 2
- Watchers: 2
- Forks: 2
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# pystreamvbyte
[](https://travis-ci.org/iiSeymour/pystreamvbyte)
Python bindings to [streamvbyte](https://github.com/lemire/streamvbyte).
## Installing
```
$ pip install --user pystreamvbyte
```
## Example
```python
>>> import numpy as np
>>> from streamvbyte import encode, decode
>>>
>>> size = int(40e6)
>>> dtype = np.uint32 # int16, uint16, int32, uint32 supported
>>> data = np.random.randint(0, 512, size=size, dtype=np.uint32)
>>> data.nbytes
160000000
>>> compressed = encode(data)
>>> compressed.nbytes
70001679
>>> recovered = decode(compressed, size, dtype=dtype)
>>> compressed.nbytes / data.nbytes * 100
43.751049375
```
## Development Quick Start
```
$ git clone --recurse-submodules https://github.com/iiSeymour/pystreamvbyte.git
$ python3 -m venv .venv
$ source .venv/bin/activate
$ make test
```