{"id":33002966,"url":"https://github.com/flanglet/kanzi-cpp","last_synced_at":"2026-04-04T18:08:34.745Z","repository":{"id":40675781,"uuid":"133185016","full_name":"flanglet/kanzi-cpp","owner":"flanglet","description":"Fast lossless data compression in C++","archived":false,"fork":false,"pushed_at":"2026-03-29T00:07:39.000Z","size":10019,"stargazers_count":210,"open_issues_count":1,"forks_count":8,"subscribers_count":7,"default_branch":"master","last_synced_at":"2026-03-29T02:58:11.316Z","etag":null,"topics":["asymmetric-numeral-systems","bwt","compression","cpp","decompression","huffman","lossless-data-compression","lz77","multithreading"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flanglet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2018-05-12T21:33:58.000Z","updated_at":"2026-03-29T00:07:42.000Z","dependencies_parsed_at":"2023-10-10T14:32:27.079Z","dependency_job_id":"f0f06bea-9ce4-4948-92d2-d20fa999ebfa","html_url":"https://github.com/flanglet/kanzi-cpp","commit_stats":null,"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"purl":"pkg:github/flanglet/kanzi-cpp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flanglet%2Fkanzi-cpp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flanglet%2Fkanzi-cpp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flanglet%2Fkanzi-cpp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flanglet%2Fkanzi-cpp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flanglet","download_url":"https://codeload.github.com/flanglet/kanzi-cpp/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flanglet%2Fkanzi-cpp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31407684,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T10:20:44.708Z","status":"ssl_error","status_checked_at":"2026-04-04T10:20:06.846Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asymmetric-numeral-systems","bwt","compression","cpp","decompression","huffman","lossless-data-compression","lz77","multithreading"],"created_at":"2025-11-13T14:00:38.903Z","updated_at":"2026-04-04T18:08:34.737Z","avatar_url":"https://github.com/flanglet.png","language":"C++","readme":"# Kanzi\n\nKanzi is a modern, modular, portable, and efficient lossless data compressor written in C++.\n\n* Modern: Kanzi implements state-of-the-art compression algorithms and is built to fully utilize multi-core CPUs via built-in multi-threading.\n* Modular: Entropy codecs and data transforms can be selected and combined at runtime to best suit the specific data being compressed.\n* Portable: Supports a wide range of operating systems, compilers, and C++ standards (details below).\n* Expandable: A clean, interface-driven design—with no external dependencies—makes Kanzi easy to integrate, extend, and customize.\n* Efficient: Carefully optimized to balance compression ratio and speed for practical, high-performance usage.\n\nUnlike most mainstream lossless compressors, Kanzi is not limited to a single compression paradigm. By combining multiple algorithms and techniques, it supports a broader range of compression ratios and adapts better to diverse data types.\n\nMost traditional compressors underutilize modern hardware by running single-threaded—even on machines with many cores. Kanzi, in contrast, is concurrent by design, compressing multiple blocks in parallel across threads for significant performance gains. However, it is not compatible with standard compression formats.\n\nIt’s important to note that Kanzi is a data compressor, not an archiver. It includes optional checksums for verifying data integrity, but does not provide features like cross-file deduplication or data recovery mechanisms. That said, it produces a seekable bitstream, meaning one or more consecutive blocks can be decompressed independently, without needing to process the entire stream.\n\nFor more details, see [Wiki](https://github.com/flanglet/kanzi-cpp/wiki), [Q\u0026A](https://github.com/flanglet/kanzi-cpp/wiki/q\u0026a) and [DeepWiki](https://deepwiki.com/flanglet/kanzi-cpp/1-overview)\n\nSee how to reuse the C and C++ APIs: [here](https://github.com/flanglet/kanzi-cpp/wiki/Using-and-extending-the-code)\n\nThere is a Java implementation available here: https://github.com/flanglet/kanzi\n\nThere is a Go implementation available here: https://github.com/flanglet/kanzi-go\n\n![Build Status](https://github.com/flanglet/kanzi-cpp/actions/workflows/c-cpp.yml/badge.svg)\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=flanglet_kanzi-cpp\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=flanglet_kanzi-cpp)\n[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=flanglet_kanzi-cpp\u0026metric=ncloc)](https://sonarcloud.io/summary/new_code?id=flanglet_kanzi-cpp)\n\u003ca href=\"https://scan.coverity.com/projects/flanglet-kanzi-cpp\"\u003e\n  \u003cimg alt=\"Coverity Scan Build Status\"\n       src=\"https://img.shields.io/coverity/scan/16859.svg\"/\u003e\n\u003c/a\u003e\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)\n[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/flanglet/kanzi-cpp)\n\n\n## Why Kanzi\n\nWhile excellent open-source compressors like zstd and brotli exist, they are primarily based on Lempel-Ziv (LZ) algorithms. Zstd, in particular, is a fantastic general-purpose choice known for its speed. However, LZ-based tools have inherent limits regarding compression ratios.\n\nKanzi offers a compelling alternative for specific high-performance scenarios:\n\n* Beyond LZ: By incorporating Burrows-Wheeler Transform (BWT) and Context Modeling (CM), Kanzi can achieve compression ratios that traditional LZ methods cannot.\n\n* Speed where it counts: While LZ is ideal for \"compress once, decompress often\" (like software distribution), it often slows down significantly at high compression settings. Kanzi leverages multi-core CPUs to maintain performance, making it highly effective for backups, real-time data generation, and one-off transfers.\n\n* Content-Aware: Kanzi features built-in, customizable transforms for specific data types (e.g., multimedia, DNA, UTF text), improving efficiency where generic compressors fail.\n\n* Extensible: The architecture is developer-friendly, making it straightforward to implement new transforms or entropy codecs for experimentation or niche data types.\n\n\n\n## Benchmarks\n\nKanzi version 2.5.0 C++ implementation\n\n_Note: The default block size at level 9 is 32MB. This limits the number of threads in use, especially with smaller files like enwik8, but all tests below are performed with default values._\n\n\n### silesia.tar\n\nTest machine:\n\nAMD Ryzen 9 9950X 16-Core Processor running Ubuntu 25.10\n\nDownload at http://sun.aei.polsl.pl/~sdeor/corpus/silesia.zip\n\n|        Compressor               |  Encoding (ms)  |  Decoding (ms)  |      Size        |\n|---------------------------------|-----------------|-----------------|------------------|\n|Original                         |                 |                 |   211,957,760    |\n|lz4 1.1.10 -T16 -4               |        18       |         13      |    79,910,851    |\n|**kanzi -l 1**                   |      **72**     |       **42**    |    79,331,051    |\n|zstd 1.5.8 -T16 -2               |         6       |         11      |    69,443,247    |\n|**kanzi -l 2**                   |      **64**     |       **42**    |    68,616,621    |\n|brotli 1.1.0 -2                  |       880       |        333      |    68,040,160    |\n|gzip 1.13 -9                     |     10328       |        704      |    67,651,076    |\n|**kanzi -l 3**                   |     **109**     |       **58**    |    63,966,794    |\n|zstd 1.5.8 -T16 -5               |       138       |        123      |    62,867,556    |\n|**kanzi -l 4**                   |     **194**     |      **102**    |    61,183,757    |\n|zstd 1.5.8 -T16 -9               |       320       |        114      |    59,233,481    |\n|brotli 1.1.0 -6                  |      4039       |        299      |    58,511,709    |\n|zstd 1.5.8 -T16 -13              |      1820       |        112      |    57,843,283    |\n|brotli 1.1.0 -9                  |     23030       |        293      |    56,407,229    |\n|bzip2 1.0.8 -9                   |      8223       |       3453      |    54,588,597    |\n|**kanzi -l 5**                   |     **529**     |      **255**    |    53,853,702    |\n|zstd 1.5.8 -T16 -19              |     11290       |        130      |    52,830,213    |\n|**kanzi -l 6**                   |     **919**     |      **532**    |    49,472,084    |\n|xz 5.8.1 -9                      |     43611       |        931      |    48,802,580    |\n|bsc 3.3.11 -T16                  |      1201       |        698      |    47,900,848    |\n|**kanzi -l 7**                   |    **1153**     |      **888**    |    47,330,422    |\n|bzip3 1.5.1.r3-g428f422 -j 16    |      2348       |       2218      |    47,260,281    |\n|**kanzi -l 8**                   |    **4473**     |     **4881**    |    42,962,913    |\n|**kanzi -l 9**                   |   **11618**     |    **12381**    |    41,520,670    |\n\n\n\n\n![Graph for Silesia on AMD Ryzen 9950X](doc/Plot_silesia.png)\n\nRound-trip graph for Silesia on AMD Ryzen 9950X (X = compTime + 2*decompTime, Y = comp size)\n\n\n### enwik8\n\nTest machine:\n\nApple M3 24 GB macOS Sonoma 15.7.3\n\nDownload at https://mattmahoney.net/dc/enwik8.zip\n\n|   Compressor    | Encoding (ms)  | Decoding (ms)  |    Size      |\n|-----------------|----------------|----------------|--------------|\n|Original         |                |                |  100,000,000 |\n|kanzi -l 1       |       139      |         85     |   42,870,183 |\n|kanzi -l 2       |       131      |         92     |   37,544,247 |\n|kanzi -l 3       |       215      |        123     |   32,551,405 |\n|kanzi -l 4       |       303      |        170     |   29,536,581 |\n|kanzi -l 5       |       670      |        372     |   26,528,254 |\n|kanzi -l 6       |      1009      |        727     |   24,076,765 |\n|kanzi -l 7       |      1607      |       1366     |   22,817,360 |\n|kanzi -l 8       |      6371      |       6752     |   21,181,992 |\n|kanzi -l 9       |      8260      |       8760     |   20,035,144 |\n\n\n![Graph for enwik8 on AMD Ryzen 9950X](doc/Plot_enwik8.png)\n\nRound-trip graph for enwik8 on AMD Ryzen 9950X  (X = compTime + 2*decompTime, Y = comp size)\n\n\n\n### More benchmarks\n\n[Comprehensive lzbench benchmarks](https://github.com/flanglet/kanzi-cpp/wiki/Performance)\n\n[More round trip scores](https://github.com/flanglet/kanzi-cpp/wiki/Round%E2%80%90trips-scores)\n\n\n## Build Kanzi\n\n* Platforms: Windows (Visual Studio), Linux, macOS, BSD\n* Dependencies: None.\n* Portability: Designed for easy porting to other OSs.\n* Multithreading: Supported by default.\n\n### Visual Studio 2008\nUnzip the file \"Kanzi_VS2008.zip\" in place.\nThe solution generates a Windows 32 binary. Multithreading is not supported with this version.\n\n### Visual Studio 2022\nUnzip the file \"Kanzi_VS2022.zip\" in place.\nThe solution generates a Windows 64 binary and library.\n\n### mingw-w64\nGo to the source directory and run 'make clean \u0026\u0026 mingw32-make.exe kanzi'. The Makefile contains\nall the necessary targets. Tested successfully on Win64 with mingw-w64 g++ 8.1.0.\nMultithreading is supported with g++ version 5.0.0 or newer.\nBuilds successfully with C++11, C++14, C++17.\n\n### Linux\nGo to the source directory and run 'make clean \u0026\u0026 make kanzi'. The Makefile contains all the necessary\ntargets. Build successfully on Ubuntu with many versions of g++ and clang++.\nMultithreading is supported with g++ version 5.0.0 or newer.\nBuilds successfully with C++98, C++11, C++14, C++17, C++20.\n\n### macOS\nGo to the source directory and run 'make clean \u0026\u0026 make kanzi'. The Makefile contains all the necessary\ntargets. Build successfully on MacOs with several versions of clang++.\nBuilds successfully with C++98, C++11, C++14, C++17, C++20.\n\n### BSD\nThe makefile uses the gnu-make syntax. First, make sure gmake is present (or install it: 'pkg install gmake').\nGo to the source directory and run 'gmake clean \u0026\u0026 gmake kanzi'. The Makefile contains all the necessary\ntargets. Builds successfully with C++98, C++11, C++14, C++17, C++20.\n\n### Makefile targets\n```\nclean:          removes objects, libraries and binaries\nkanzi:          builds the kanzi executable\nkanzi_static:   builds a statically linked executable\nkanzi_dynamic:  builds a dynamically linked executable\nlib:            builds static and dynamic libraries\ntest:           builds test binaries\nall:            kanzi + kanzi_static + kanzi_dynamic + lib + test\ninstall:        installs libraries, headers and executable\nuninstall:      removes installed libraries, headers and executable\n```\n\nFor those who prefer cmake, run the following commands from the top directory:\n```\nmkdir build\ncd build\ncmake ..\nmake\nctest\n```\nBy default, the cmake build generates a dynamically linked executable.\nChoose ```make kanzi_static``` to build a statically linked executable.\n\nCredits\n\nMatt Mahoney,\nYann Collet,\nJan Ondrus,\nYuta Mori,\nIlya Muravyov,\nNeal Burns,\nFabian Giesen,\nJarek Duda,\nIlya Grebnov\n\nDisclaimer\n\nUse at your own risk. Always keep a copy of your original files.\n\n","funding_links":[],"categories":["Compression"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflanglet%2Fkanzi-cpp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflanglet%2Fkanzi-cpp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflanglet%2Fkanzi-cpp/lists"}