https://github.com/leonwind/duckdb-adaptive-compression
Code for our paper "Adaptive Compression For Databases" (EDBT '24)
https://github.com/leonwind/duckdb-adaptive-compression
Last synced: 10 months ago
JSON representation
Code for our paper "Adaptive Compression For Databases" (EDBT '24)
- Host: GitHub
- URL: https://github.com/leonwind/duckdb-adaptive-compression
- Owner: leonwind
- License: mit
- Created: 2023-01-04T11:45:16.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2023-11-27T14:40:04.000Z (over 2 years ago)
- Last Synced: 2025-04-30T20:02:41.763Z (about 1 year ago)
- Language: C++
- Homepage: https://openproceedings.org/2024/conf/edbt/paper-43.pdf
- Size: 73.5 MB
- Stars: 4
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# AdaCom: Adaptive Compression For Databases
This is our implementation of our paper "Adaptive Compression For Databases" (EDBT '24), where we adaptively compress infrequently accessed column segments using succinct encoding, in [DuckDB](https://github.com/duckdb/duckdb).
## Installation
Our implementation uses the [Succinct Data Structure Library (SDSL)](https://github.com/simongog/sdsl-lite/), and it can be installed with
```shell
./installSDSL.sh
```
Afterwards, DuckDB with our extension can be compiled with:
```shell
mkdir build
cd build
cmake -DBUILD_BENCHMARKS=1 -DBUILD_TPCH_EXTENSION=1 -GNinja -DCMAKE_BUILD_TYPE=Release ..
ninja
```
## Benchmarks
Our benchmarks are in `/benchmark/micro/succinct` and can be run from the build directory with e.g.
```shell
benchmark/benchmark_runner SuccinctZipfDistribution
```
### Build issue: Linking SDSL with DuckDB
Some systems may experience issues linking the SDSL with DuckDB due to the error
```
can not be used when making a shared object; recompile with -fPIC
```
One fix is by adding the flag `-DCMAKE_POSITION_INDEPENDENT_CODE=ON` in `third_party/sdsl-lite/install.sh` in the line
```
cmake -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_INSTALL_PREFIX="${SDSL_INSTALL_PREFIX}" .. # run cmake
```
resulting in
```
cmake -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_INSTALL_PREFIX="${SDSL_INSTALL_PREFIX}" .. # run cmake
```
After adding the flag, re-running the `install.sh` script is necessary:
```
cd third_party/sdsl-lite
./install.sh ../sdsl
```