{"id":46033111,"url":"https://github.com/flanglet/kanzi-go","last_synced_at":"2026-03-01T04:25:40.734Z","repository":{"id":49719500,"uuid":"133183686","full_name":"flanglet/kanzi-go","owner":"flanglet","description":"Fast lossless data compression in Go","archived":false,"fork":false,"pushed_at":"2026-02-27T22:12:18.000Z","size":5410,"stargazers_count":616,"open_issues_count":0,"forks_count":15,"subscribers_count":8,"default_branch":"master","last_synced_at":"2026-02-28T03:21:57.142Z","etag":null,"topics":["asymmetric-numeral-systems","bwt","compression","concurrency","decompression","go","golang","huffman","lossless-data-compression","lz77"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flanglet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2018-05-12T21:10:43.000Z","updated_at":"2026-02-27T22:12:21.000Z","dependencies_parsed_at":"2023-10-22T14:28:12.600Z","dependency_job_id":"e89c8e78-dfd3-4033-ae16-7e58c487fe83","html_url":"https://github.com/flanglet/kanzi-go","commit_stats":null,"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"purl":"pkg:github/flanglet/kanzi-go","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flanglet%2Fkanzi-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flanglet%2Fkanzi-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flanglet%2Fkanzi-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flanglet%2Fkanzi-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flanglet","download_url":"https://codeload.github.com/flanglet/kanzi-go/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flanglet%2Fkanzi-go/sbom","scorecard":{"id":402537,"data":{"date":"2025-08-11","repo":{"name":"github.com/flanglet/kanzi-go","commit":"43ad809fa156875ce55509f1e87a64a377278a7b"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":5.9,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/29 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":10,"reason":"30 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: SECURITY.md:1","Info: Found linked content: SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1","Info: Found text in security policy: SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Info: jobLevel 'actions' permission set to 'read': .github/workflows/codeql.yml:30","Info: jobLevel 'contents' permission set to 'read': .github/workflows/codeql.yml:31","Info: topLevel permissions set to 'read-all': .github/workflows/codeql.yml:23","Warn: no topLevel permission defined: .github/workflows/go.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":4,"reason":"dependency not pinned by hash detected -- score normalized to 4","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/go.yml:14: update your workflow using https://app.stepsecurity.io/secureworkflow/flanglet/kanzi-go/go.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/go.yml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/flanglet/kanzi-go/go.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/go.yml:27: update your workflow using https://app.stepsecurity.io/secureworkflow/flanglet/kanzi-go/go.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/go.yml:30: update your workflow using https://app.stepsecurity.io/secureworkflow/flanglet/kanzi-go/go.yml/master?enable=pin","Info:   3 out of   7 GitHub-owned GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":10,"reason":"SAST tool is run on all commits","details":["Info: SAST configuration detected: CodeQL","Info: all commits (2) are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-18T20:24:03.151Z","repository_id":49719500,"created_at":"2025-08-18T20:24:03.151Z","updated_at":"2025-08-18T20:24:03.151Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29960248,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-01T01:47:18.291Z","status":"online","status_checked_at":"2026-03-01T02:00:07.437Z","response_time":124,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asymmetric-numeral-systems","bwt","compression","concurrency","decompression","go","golang","huffman","lossless-data-compression","lz77"],"created_at":"2026-03-01T04:25:38.558Z","updated_at":"2026-03-01T04:25:40.726Z","avatar_url":"https://github.com/flanglet.png","language":"Go","readme":"# Kanzi\n\nKanzi is a modern, modular, portable, and efficient lossless data compressor written in Go.\n\n* Modern: Kanzi implements state-of-the-art compression algorithms and is built to fully utilize multi-core CPUs via built-in multi-threading.\n* Modular: Entropy codecs and data transforms can be selected and combined at runtime to best suit the specific data being compressed.\n* Expandable: A clean, interface-driven design—with no external dependencies—makes Kanzi easy to integrate, extend, and customize.\n* Efficient: Carefully optimized to balance compression ratio and speed for practical, high-performance usage.\n\nUnlike most mainstream lossless compressors, Kanzi is not limited to a single compression paradigm. By combining multiple algorithms and techniques, it supports a broader range of compression ratios and adapts better to diverse data types.\n\nMost traditional compressors underutilize modern hardware by running single-threaded—even on machines with many cores. Kanzi, in contrast, is concurrent by design, compressing multiple blocks in parallel across threads for significant performance gains. However, it is not compatible with standard compression formats.\n\nIt’s important to note that Kanzi is a data compressor, not an archiver. It includes optional checksums for verifying data integrity, but does not provide features like cross-file deduplication or data recovery mechanisms. That said, it produces a seekable bitstream—meaning one or more consecutive blocks can be decompressed independently, without needing to process the entire stream.\n\nFor more details, see [Wiki](https://github.com/flanglet/kanzi-go/wiki), [Q\u0026A](https://github.com/flanglet/kanzi-go/wiki/q\u0026a) and [DeepWiki](https://deepwiki.com/flanglet/kanzi-go/1-overview)\n\nSee how to reuse the code here: https://github.com/flanglet/kanzi-go/wiki/Using-and-extending-the-code\n\nThere is a C++ implementation available here: https://github.com/flanglet/kanzi-cpp\n\nThere is Java implementation available here: https://github.com/flanglet/kanzi\n\n\n\n![Build Status](https://github.com/flanglet/kanzi-go/actions/workflows/go.yml/badge.svg)\n[![Go Report Card](https://goreportcard.com/badge/github.com/flanglet/kanzi-go/v2)](https://goreportcard.com/report/github.com/flanglet/kanzi-go/v2)\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=flanglet_kanzi-go\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=flanglet_kanzi-go)\n[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=flanglet_kanzi-go\u0026metric=ncloc)](https://sonarcloud.io/summary/new_code?id=flanglet_kanzi-go)\n[![Documentation](https://godoc.org/github.com/flanglet/kanzi-go?status.svg)](http://godoc.org/github.com/flanglet/kanzi-go/v2)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)\n[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/flanglet/kanzi-go)\n\n## Why Kanzi\n\nWhile excellent open-source compressors like zstd and brotli exist, they are primarily based on Lempel-Ziv (LZ) algorithms. Zstd, in particular, is a fantastic general-purpose choice known for its speed. However, LZ-based tools have inherent limits regarding compression ratios.\n\nKanzi offers a compelling alternative for specific high-performance scenarios:\n\n* Beyond LZ: By incorporating Burrows-Wheeler Transform (BWT) and Context Modeling (CM), Kanzi can achieve compression ratios that traditional LZ methods cannot.\n\n* Speed where it counts: While LZ is ideal for \"compress once, decompress often\" (like software distribution), it often slows down significantly at high compression settings. Kanzi leverages multi-core CPUs to maintain performance, making it highly effective for backups, real-time data generation, and one-off transfers.\n\n* Content-Aware: Kanzi features built-in, customizable transforms for specific data types (e.g., multimedia, DNA, UTF text), improving efficiency where generic compressors fail.\n\n* Extensible: The architecture is developer-friendly, making it straightforward to implement new transforms or entropy codecs for experimentation or niche data types.\n\n\n\n## Benchmarks\n\nKanzi version 2.5.0\n\nGo version 1.26.0\n\n### silesia.tar\n\nTest machine: AMD Ryzen 9950X on Ubuntu 25.10\n\nDownload at http://sun.aei.polsl.pl/~sdeor/corpus/silesia.zip\n\n|        Compressor               |  Encoding (ms)  |  Decoding (ms)  |    Size          |\n|---------------------------------|-----------------|-----------------|------------------|\n|Original                         |                 |                 |   211,957,760    |\n|lz4 1.1.10 -T16 -4               |        18       |         13      |    79,910,851    |\n|**kanzi -l 1**                   |     **173**     |       **75**    |    79,331,051    |\n|zstd 1.5.8 -T16 -2               |         6       |         11      |    69,443,247    |\n|**kanzi -l 2**                   |     **137**     |       **88**    |    68,616,621    |\n|brotli 1.1.0 -2                  |       880       |        333      |    68,040,160    |\n|gzip 1.13 -9                     |     10328       |        704      |    67,651,076    |\n|**kanzi -l 3**                   |     **230**     |      **112**    |    63,966,794    |\n|zstd 1.5.8 -T16 -5               |       138       |        123      |    62,867,556    |\n|**kanzi -l 4**                   |     **448**     |      **185**    |    61,183,757    |\n|zstd 1.5.8 -T16 -9               |       320       |        114      |    59,233,481    |\n|brotli 1.1.0 -6                  |      4039       |        299      |    58,511,709    |\n|zstd 1.5.8 -T16 -13              |      1820       |        112      |    57,843,283    |\n|brotli 1.1.0 -9                  |     23030       |        293      |    56,407,229    |\n|bzip2 1.0.8 -9                   |      8223       |       3453      |    54,588,597    |\n|**kanzi -l 5**                   |     **940**     |      **398**    |    53,853,702    |\n|zstd 1.5.8 -T16 -19              |     11290       |        130      |    52,830,213    |\n|**kanzi -l 6**                   |    **1255**     |      **973**    |    49,472,084    |\n|xz 5.8.1 -9                      |     43611       |        931      |    48,802,580    |\n|bsc 3.3.11 -T16                  |      1201       |        698      |    47,900,848    |\n|**kanzi -l 7**                   |    **1714**     |     **1227**    |    47,330,422    |\n|bzip3 1.5.1.r3-g428f422 -j 16    |      2348       |       2218      |    47,260,281    |\n|**kanzi -l 8**                   |    **8362**     |     **7943**    |    42,962,913    |\n|**kanzi -l 9**                   |   **17944**     |    **17512**    |    41,520,670    |\n\n\n\n\n### enwik8\n\nTest machine: Apple M3 24 GB Sonoma 15.7.3\n\nDownload at https://mattmahoney.net/dc/enwik8.zip\n\n|   Compressor    | Encoding (ms)  | Decoding (ms)  |  Size        |\n|-----------------|----------------|----------------|--------------|\n|Original         |                |                |  100,000,000 |\n|Kanzi -l 1       |       146      |         62     |   42,870,183 |\n|Kanzi -l 2       |       162      |         75     |   37,544,247 |\n|Kanzi -l 3       |       262      |        134     |   32,551,405 |\n|Kanzi -l 4       |       422      |        191     |   29,536,581 |\n|Kanzi -l 5       |       789      |        415     |   26,528,254 |\n|Kanzi -l 6       |      1261      |       1317     |   24,076,765 |\n|Kanzi -l 7       |      2101      |       2183     |   22,817,360 |\n|Kanzi -l 8       |      7874      |       7890     |   21,181,992 |\n|Kanzi -l 9       |     11124      |      11059     |   20,035,144 |\n\n\n\n\n# Build\n\nUsing formal releases is recommended (see https://github.com/flanglet/kanzi-go/releases).\n```\ngo install github.com/flanglet/kanzi-go/v2/app@v2.5.0\n```\n\nOtherwise, to build manually from the latest tag, follow the instructions below:\n\n```\ngit clone https://github.com/flanglet/kanzi-go.git\n\ncd kanzi-go/v2/app\n\ngo build Kanzi.go BlockCompressor.go BlockDecompressor.go InfoPrinter.go\n```\n\nThe bistream is backward compatible, however, the guarantee only applies to releases. Users can expect incompatibilities or breakage due to bitstream changes in between releases.\n\nCredits\n\nMatt Mahoney,\nYann Collet,\nJan Ondrus,\nYuta Mori,\nIlya Muravyov,\nNeal Burns,\nFabian Giesen,\nJarek Duda,\nIlya Grebnov\n\nDisclaimer\n\nUse at your own risk. Always keep a copy of your original files.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflanglet%2Fkanzi-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflanglet%2Fkanzi-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflanglet%2Fkanzi-go/lists"}