Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kspalaiologos/qbdiff
building and applying patches to binary files
https://github.com/kspalaiologos/qbdiff
Last synced: about 2 months ago
JSON representation
building and applying patches to binary files
- Host: GitHub
- URL: https://github.com/kspalaiologos/qbdiff
- Owner: kspalaiologos
- License: lgpl-3.0
- Created: 2023-01-16T16:38:06.000Z (over 1 year ago)
- Default Branch: trunk
- Last Pushed: 2023-11-01T02:03:35.000Z (11 months ago)
- Last Synced: 2024-05-27T21:41:02.989Z (4 months ago)
- Language: C
- Homepage:
- Size: 133 KB
- Stars: 65
- Watchers: 4
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# qbdiff
[![Build](https://github.com/kspalaiologos/qbdiff/actions/workflows/build.yml/badge.svg)](https://github.com/kspalaiologos/qbdiff/actions/workflows/build.yml)
Quick Binary Diff. An improved and modernised version of bsdiff. An overview:
- Faster than bsdiff, can take advantage of multiple CPU cores.
- Generally smaller patch files than bsdiff, supports only LZMA compression which performs excellently on sparse binary data.
- Requires less memory than bsdiff.
- Based on Colin Percival's algorithm used in bsdiff.
- Integrity checking using BLAKE2b.
- Can be used as a library.## Installation
Make sure to obtain `libomp-dev` and `liblzma-dev`. While OpenMP is not necessary, it will decrease the wall clock time taken by the program approximately threefold.
```
# If using a git clone (not needed for source packages), first...
$ ./bootstrap.sh# All...
$ ./configure
$ make
$ sudo make install
```## Use cases
I initially created qbdiff to enable certain kinds of applications (e.g. games) to allow easy automatic updates. I've also thought about the idea of binary diffing in the context of binary linux distributions, which ship patches to software in this (rather succinct) format.
## Benchmarks against bsdiff 4.3
```
bsdiff 1g: 229.83s +/- 5s
qbdiff 1g: 46s +/- 1sbsdiff 1m: 87.5ms +/- 2ms
qbdiff 1m: 37ms +/- 1msbsdiff 10k: 1ms +/- 0.5ms
qbdiff 10k: 2.8ms +/- 1msbsdiff 10m: 1.1s +/- 0.05s
qbdiff 10m: 369ms +/- 1msbsdiff 100k: 11ms +/- 0.1ms
qbdiff 100k: 8ms +/- 2msbsdiff 100m: 16.3s +/- 100ms
qbdiff 100m: 4.5s +/- 20ms
```qbdiff uses LZMA, which has considerably longer compression time which is responsible for great compression ratio (considerably better than bzip2 on binary data) and fast decompression speed, meaning that (compared to bsdiff's bzip2), patch creation is the bottleneck where it is supposed to be slower. However, when it comes to decompression, bzip2 is a lot slower taking usually two to six times more time than LZMA ([source](https://tukaani.org/lzma/benchmarks.html)).
Furthermore, the [bsdiff memory usage](https://www.daemonology.net/bsdiff/) is `max(17*n,9*n+m)+O(1)`, while qbdiff memory usage is usually bounded by `O(5*n+m)+O(1)` in most use cases - the improvement is clear and almost twofold-threefold.
## Thanks
- Ilya Grebnov for libsais
- Igor Pavlov & Lasse Collin for liblzma
- Colin Percival for bsdiff
- Benjamin Strachan for conducting the performance benchmarks.