{"id":26947139,"url":"https://github.com/lsds/multikernelboss","last_synced_at":"2025-04-02T20:18:05.638Z","repository":{"id":212647658,"uuid":"731100386","full_name":"lsds/MultiKernelBOSS","owner":"lsds","description":null,"archived":false,"fork":false,"pushed_at":"2023-12-15T11:12:39.000Z","size":109,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-03-27T08:48:29.819Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lsds.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-12-13T11:04:03.000Z","updated_at":"2024-01-23T13:53:57.000Z","dependencies_parsed_at":"2023-12-15T12:47:50.309Z","dependency_job_id":null,"html_url":"https://github.com/lsds/MultiKernelBOSS","commit_stats":null,"previous_names":["lsds/multikernelboss"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsds%2FMultiKernelBOSS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsds%2FMultiKernelBOSS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsds%2FMultiKernelBOSS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsds%2FMultiKernelBOSS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lsds","download_url":"https://codeload.github.com/lsds/MultiKernelBOSS/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246884767,"owners_count":20849554,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-02T20:18:04.786Z","updated_at":"2025-04-02T20:18:05.620Z","avatar_url":"https://github.com/lsds.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multi-Kernel BOSS Benchmarks\n\n## Pre-requisites:\n* NVIDIA driver\n* Docker\n\n## Preparation on the host for CUDA support\n```\nsudo apt update\nsudo apt install nvidia-container-toolkit\nsudo nvidia-ctk runtime configure --runtime=docker\nsudo systemctl restart docker\n```\n\n## Run the container image\n```\ndocker run -ti --runtime=nvidia --gpus all --workdir / ghcr.io/hmohrdaurat/buildenvironment/velox-docker-image:latest\n```\n\n## Download and install CUDA 12\n\u003e only the CUDA Toolkit must be installed, not the driver\n```\n./cuda_12.0.1_525.85.12_linux.run\nPATH=$PATH:/usr/local/cuda-12.0/bin\nldconfig\n```\n\n## Get the benchmark code\n```\ngit clone https://gitfront.io/r/hmohrdaurat/HPqpn35ChFLR/MultiKernelBOSS-VLDB-Code.git\ncd MultiKernelBOSS-VLDB-Code\n```\n\n## Generate TPC-H dataset\n```\nchmod u+x generate_tpch_data.sh\n./generate_tpch_data.sh\n```\n\n* Generate TPC-H for SF 1 only:\n```\n./generate_tpch_data.sh 1 1\n```\n\n## Compile the code\n```\nmkdir build\ncd build\ncmake -DCMAKE_INSTALL_PREFIX:PATH=.. -DCMAKE_C_COMPILER=clang-14 -DCMAKE_CXX_COMPILER=clang++-14 -DCMAKE_BUILD_TYPE=Release -DVELOX_DIRS=/usr/local/velox -B. ..\ncd ..\ncmake --build build --target install\n```\n\n## Run the benchmarks\n```\ncd bin\n```\n\n* Velox with SF 1:\n```\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --disable-constraints --library libBOSSArrowStorage.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q1/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --disable-constraints --library libBOSSArrowStorage.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q3/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --disable-constraints --library libBOSSArrowStorage.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --disable-constraints --library libBOSSArrowStorage.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q9/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --disable-constraints --library libBOSSArrowStorage.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q18/BOSS/1000MB\n```\n\n* MonetDB and DuckDB with SF 1:\n```\nLD_LIBRARY_PATH=../lib ./Benchmarks --fixed-point-numeric-type --benchmark_filter=TPC-H_Q[0-9]+/MonetDB/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --fixed-point-numeric-type --benchmark_filter=TPC-H_Q[0-9]+/DuckDB/1000MB\n```\n\n* BOSS (CPU) with SF 1:\n```\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineCPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q1/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineCPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q3V_POST-FILTER-2JOINS/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineCPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineCPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q9V_POST-FILTER-AND-PRIORITY/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineCPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q18/BOSS/1000MB\n```\n\n* BOSS (GPU) with SF 1:\n```\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q1/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q3V_POST-FILTER-2JOINS/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q9V_POST-FILTER-AND-PRIORITY/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q18/BOSS/1000MB\n```\n\n* ArrayFire partial evaluation with SF 1:\n```\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --benchmark_filter=TPC-H_Q1V_POST-FILTER/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --benchmark_filter=TPC-H_Q3V_POST-FILTER-2JOINS/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --benchmark_filter=TPC-H_Q9V_POST-FILTER-AND-PRIORITY/BOSS/1000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --benchmark_filter=TPC-H_Q18/BOSS/1000MB\n```\n\n* BOSS (GPU) with SF 100:\n```\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 12000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q1/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 8500 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q3V_POST-FILTER-1JOIN/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 11000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 12000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q9V_POST-FILTER-AND-PRIORITY/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 12000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q18/100000MB\n```\n\n* BOSS (GPU) with various maximum allowed GPU memory on TPC-H Q6 with SF 100:\n```\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 0 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 1000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 2000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 3000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 4000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 5000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 6000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 7000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 8000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 9000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 10000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --max-gpu-memory-cache 11000 --all-strings-as-integers --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6V_NESTED-SELECT2/100000MB\n```\n\n## Ablation Study\n\n* BOSS (GPU) with SF 10 (with data copy from Storage to ArrayFire):\n```\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-in --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q1/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-in --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q3V_POST-FILTER-2JOINS/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-in --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-in --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q9V_POST-FILTER-AND-PRIORITY/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-in --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q18/10000MB\n```\n\n* BOSS (GPU) with SF 10 (with data copy from ArrayFire To Velox):\n```\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-out --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q1/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-out --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q3V_POST-FILTER-2JOINS/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-out --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-out --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q9V_POST-FILTER-AND-PRIORITY/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-out --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q18/10000MB\n```\n\n* BOSS (GPU) with SF 10 (with all data copies):\n```\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-in --benchmark-data-copy-out --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q1/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-in --benchmark-data-copy-out --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q3V_POST-FILTER-2JOINS/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-in --benchmark-data-copy-out --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q6/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-in --benchmark-data-copy-out --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q9V_POST-FILTER-AND-PRIORITY/10000MB\nLD_LIBRARY_PATH=../lib ./Benchmarks --default-storage-block-size 268435456 --all-strings-as-integers --benchmark-data-copy-in --benchmark-data-copy-out --library libBOSSArrowStorage.so --library libBOSSArrayFireEngineGPU.so --library libBOSSVeloxEngine.so --benchmark_filter=TPC-H_Q18/10000MB\n```\n\n* BOSS without fast path:\n\nrequires to re-compile the code with the line 21 in BOSS/Source/Expression.hpp modified as:\n```\n#define ABLATION_NO_FAST_PATH\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flsds%2Fmultikernelboss","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flsds%2Fmultikernelboss","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flsds%2Fmultikernelboss/lists"}