https://github.com/xermicus/floatpack
Lossless Compression for precission floats
https://github.com/xermicus/floatpack
Last synced: 8 months ago
JSON representation
Lossless Compression for precission floats
- Host: GitHub
- URL: https://github.com/xermicus/floatpack
- Owner: xermicus
- Created: 2020-03-02T20:48:26.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2021-11-12T16:14:59.000Z (over 4 years ago)
- Last Synced: 2025-02-24T13:23:48.560Z (over 1 year ago)
- Language: Rust
- Homepage:
- Size: 10.7 KB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# floatpack
Bitpacking with SIMD for `Decimal` from the `rust_decimal` crate.
# In a nutshell
The algorithm:
1. Each `Decimal` value is serialized into its components (= 4 x `u32`)
2. The resulting 4 component streams are individually compressed by storing their cumulative difference (XOR)
3. The 4 compressed component streams are then bit-packed
The idea is that this should get good compression rates with little computational complexity, especially for contiguous data. In timeseries data, usually:
* one datapoint only differs slightly from the next one in the series and
* you have a lot of datapoints
This represents a highly specific use-case. If you are not dealing with timeseries data other compression algorithms are probably more suitable.
# Usage example
``` rust
use floatpack::{pack, unpack};
use rust_decimal_macros::*;
let values = vec![dec!(1.0), dec!(2.0), dec!(3.0)];
assert_eq!(values, unpack(&pack(&values[..])));
```