https://github.com/freaky/compresstimator
Simple and fast compressibility tester
https://github.com/freaky/compresstimator
Last synced: 10 months ago
JSON representation
Simple and fast compressibility tester
- Host: GitHub
- URL: https://github.com/freaky/compresstimator
- Owner: Freaky
- License: mit
- Created: 2019-06-21T15:29:14.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2020-04-24T02:51:10.000Z (about 6 years ago)
- Last Synced: 2025-05-06T16:49:01.019Z (about 1 year ago)
- Language: Rust
- Size: 11.7 KB
- Stars: 11
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# Compresstimator
[](https://crates.io/crates/compresstimator)
A simple vaguely statistical file compressibility tester.
## Description
`Compresstimator` is a library for quickly determining the rough compressibility
of a given file, using linear sampling (i.e. testing every n-th block) and lz4
compression.
For large files this can give a reasonable idea of compressibility with just a
few dozen seeks and a few hundred kilobytes of data - usually serviced by even
slow hard disks in under a second.
## Usage
`Compresstimator` consists of a public type that encapsulates the settings of
the estimator - currently just the block size used for sampling. In future this
may also include settings for trading accuracy for speed.
All estimation functions return an `f32` indicating the compression ratio,
between 0 and 1.
```rust
use compresstimator::Compresstimator;
// Create an estimator with the default block size of 4096 bytes
let estimator = Compresstimator::new();
// This is usually accurate to within 10-15% for lz4 level 1
if estimator.compresstimate_file("huge_file")? > 0.95 {
println!("Probably doesn't compress well.");
}
// If you have the file size, you can avoid a metadata lookup
let len = fs::metadata("huge_file")?.len();
if estimator.compresstimate_file_len("huge_file", len)? > 0.95 {
println!("Probably doesn't compress well.");
}
// You can also pass in a handle to any Read + Seek
let mut f = File::open("huge_file")?;
if estimator.compresstimate(&mut f, len)? > 0.95 {
println!("Probably doesn't compress well.");
}
```