{"id":21655284,"url":"https://github.com/pseitz/ans_flex","last_synced_at":"2025-03-20T04:44:07.330Z","repository":{"id":66217831,"uuid":"333706207","full_name":"PSeitz/ans_flex","owner":"PSeitz","description":null,"archived":false,"fork":false,"pushed_at":"2023-11-26T02:19:35.000Z","size":356,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-25T06:25:37.651Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PSeitz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-28T09:17:50.000Z","updated_at":"2023-05-31T16:43:10.000Z","dependencies_parsed_at":"2023-11-26T03:22:40.358Z","dependency_job_id":"7a534cda-1248-4d0a-8fb7-4d84cd1d9263","html_url":"https://github.com/PSeitz/ans_flex","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Fans_flex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Fans_flex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Fans_flex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Fans_flex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PSeitz","download_url":"https://codeload.github.com/PSeitz/ans_flex/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244554125,"owners_count":20471173,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-25T08:31:17.131Z","updated_at":"2025-03-20T04:44:07.308Z","avatar_url":"https://github.com/PSeitz.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ans_flex\n\nans_flex is a FSE/ANS implementation in Rust, a compressor in the family of entropy encoders (statistical compression).\n\nFSE ([Finite State Entropy](https://github.com/Cyan4973/FiniteStateEntropy/)) is a ANS variant from Yann Collet. Main advantage is, that it requires only additions,\nmasks, and shifts.\n\nANS (Asymetric Numeral Systems) was introduced by Jarek Duda and is a popular compression standard\nused in compression algorithms like zstd, due to its high compression ration and reasonable\ncompression speed. In comparison to huffman it has the advantage to using fractional bits, when encoding symbols.\n\nIf you want a better understanding of ANS, I can recommend \"Understanding Compression\" by Colton\nMcAnlis and Aleks Haecky as the foundation and then diving into the blog posts of [Charles Bloom](http://cbloomrants.blogspot.com/2014/01/1-30-14-understanding-ans-1.html)\nand [Yann Collet](https://fastcompression.blogspot.com/2013/12/finite-state-entropy-new-breed-of.html)\nThe [ANS paper](https://arxiv.org/pdf/1311.2540.pdf) from Jarek Duda is also interesting, but without a solid\nfoundation in math and compression it will be difficult to follow.\n\nNote, that entropy compression like ans is usually not purely used own its own, but in conjunction with other compression techniques like Lempel-Ziv.\n\n# Performance\n\nPerformance seems to be slightly faster than https://github.com/Cyan4973/FiniteStateEntropy/, which uses a close variant of its fse in zstd.\n\n`cargo bench`\n\n```\n\ncompression/ans_flex/66675                                                                            \n                        time:   [176.11 us 176.37 us 176.66 us]\n                        thrpt:  [359.93 MiB/s 360.52 MiB/s 361.05 MiB/s]\n                 change:\n                        time:   [-12.125% -11.822% -11.525%] (p = 0.00 \u003c 0.05)\n                        thrpt:  [+13.026% +13.407% +13.798%]\n                        Performance has improved.\nFound 5 outliers among 100 measurements (5.00%)\n  5 (5.00%) high mild\n\ndecompression/deans_flex_reuse/66675                                                                            \n                        time:   [110.24 us 110.95 us 111.84 us]\n                        thrpt:  [568.53 MiB/s 573.08 MiB/s 576.80 MiB/s]\n                 change:\n                        time:   [-4.0021% -3.4458% -2.9494%] (p = 0.00 \u003c 0.05)\n                        thrpt:  [+3.0390% +3.5688% +4.1689%]\n                        Performance has improved.\nFound 5 outliers among 100 measurements (5.00%)\n  3 (3.00%) high mild\n  2 (2.00%) high severe\n\n\n➜  programs git:(dev) ✗ ./fse -e -b ../../ans_flex/benches/compression_66k_JSON.txt\nFSE : Finite State Entropy, 64-bits demo by Yann Collet (Feb 19 2021)\nsion_66k_JSON.txt :     66675 -\u003e     43661 (65.48%),  366.6 MB/s ,  469.9 MB/s\n\n\nhttps://github.com/Cyan4973/FiniteStateEntropy/\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpseitz%2Fans_flex","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpseitz%2Fans_flex","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpseitz%2Fans_flex/lists"}