{"id":33050924,"url":"https://github.com/powturbo/Turbo-Run-Length-Encoding","last_synced_at":"2025-11-18T20:02:20.066Z","repository":{"id":27642746,"uuid":"31127527","full_name":"powturbo/Turbo-Run-Length-Encoding","owner":"powturbo","description":"TurboRLE-Fastest Run Length Encoding","archived":false,"fork":false,"pushed_at":"2023-04-07T11:28:19.000Z","size":15897,"stargazers_count":287,"open_issues_count":2,"forks_count":27,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-03-21T04:41:35.672Z","etag":null,"topics":["avx2","benchmark","c","c-plus-plus","java","rle","run-length-encoding","simd","sse","sse2"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/powturbo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2015-02-21T14:13:09.000Z","updated_at":"2025-01-30T20:35:09.000Z","dependencies_parsed_at":"2024-01-16T23:26:41.318Z","dependency_job_id":"a427d927-cac1-4710-8721-6dc49407f9c1","html_url":"https://github.com/powturbo/Turbo-Run-Length-Encoding","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/powturbo/Turbo-Run-Length-Encoding","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/powturbo%2FTurbo-Run-Length-Encoding","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/powturbo%2FTurbo-Run-Length-Encoding/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/powturbo%2FTurbo-Run-Length-Encoding/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/powturbo%2FTurbo-Run-Length-Encoding/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/powturbo","download_url":"https://codeload.github.com/powturbo/Turbo-Run-Length-Encoding/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/powturbo%2FTurbo-Run-Length-Encoding/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":285129556,"owners_count":27119601,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-18T02:00:05.759Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["avx2","benchmark","c","c-plus-plus","java","rle","run-length-encoding","simd","sse","sse2"],"created_at":"2025-11-14T03:00:23.119Z","updated_at":"2025-11-18T20:02:20.058Z","avatar_url":"https://github.com/powturbo.png","language":"C","funding_links":[],"categories":["压缩"],"sub_categories":[],"readme":"## TurboRLE: Turbo Run Length Encoding\n[![Build ubuntu](https://github.com/powturbo/Turbo-Run-Length-Encoding/actions/workflows/build.yaml/badge.svg)](https://github.com/powturbo/Turbo-Run-Length-Encoding/actions/workflows/build.yaml)\n\n##### Efficient and fastest **Run Length Encoding** library\n  - ARM NEON support\n  - 100% C (C++ compatible headers), without inline assembly\n  - Most efficient compression \n  - No other RLE compress or decompress faster with better compression\n  - :sparkles: faster compression and 2x faster decompression with :+1: SIMD\n  - :+1: **Java** Critical Natives Interface. Access TurboRLE **incl. SIMD!** from Java\n  - Compress better and up to 12 times faster and decompress up to 6 times faster than other fast RLEs\n  - Can be faster than memcpy!\n  - :+1: **ZERO!** byte overhead\n\n  - No modification of the raw data, preserving compressibility for further postprocessing (ex. entropy coding)\n  - Order preserving \n\n##### TurboRLE\n  - **TRLE**: TurboRLE - Efficient and fast Run Length Encoding\n  - **SRLE**: TurboRLE Escape - Fast Run Length Encoding with automatic escape determination \n\n## Benchmark:\n- Single thread with [TurboBench](https://github.com/powturbo/TurboBench)\n- Realistic and practical benchmark with large files and different distributions\n\n###### External functions benchmarked\n  - **[MRLE](http://encode.su/threads/2121-No-more-encoding-overhead-in-Run-Length-Encoding-Read-about-Mespotine-RLE-here-)**: Mespotine RLE: Run Length Encoding\n  - **[RLE8](https://github.com/rainerzufalldererste/rle8)**: A fast 8 bit Run Length Encoding (SSE/AVX2)\n------------------------------------------------------------------------\n#### CPU: Skylake i7-6700 3.4GHz, gcc 8.3 (2019-08)\n- BMP File: [girl.bmp in RLE64Samples](http://sourceforge.net/projects/nikkhokkho/files/RLE64/3.00/)\n\n(bold = pareto)  MB=1.000.000\n\n|C Size|ratio%|C MB/s|D MB/s|Name / 2019-08|\n|--------:|-----:|--------:|--------:|----------------|\n|2623680|  0.6|**2089**|**11385**|**trle**|\n|4148455|  1.0|**2095**|**12521**|**srle 0** (auto escape)|\n|4482384|  1.1|378|**12302**|**mrle**|\n|4744806|  1.2|**11307**|**13205**|**srle 8**|\n|5901235|  1.5|890 |11998|   rle8 1|\n|8431844|  2.1|7368|**12693**|**srle 16**|\n|13722311|  3.4|**11090**|**13188**|**srle 32**|\n|19839711|  4.9|**16269**|**13733**|**srle 64**|\n|403920054|100.0|13978|**14001**|**memcpy**|\n\n- Checkers program \"End Game Table Base\": [1034.db](http://encode.su/threads/2077-EGTB-compression?p=41392\u0026viewfull=1#post41392)\n\n|C Size|ratio%|C MB/s|D MB/s|Name / 2019-08|\n|--------:|-----:|--------:|--------:|----------------|\n|73108990| 17.4|**774**|**3020**|**trle**|\n|84671759| 20.2|764|**5113**|**srle 0**|\n|88055360| 21.0| 262|1491|   mrle|\n|88666372| 21.2| 456| 2669| rle8 1|\n|92369164| 22.0|**1101**|**5967**|**srle 8**|\n|113561548| 27.1|**2028**|**7114**|**srle 16**|\n|136918311| 32.7|**3588**|**9026**|**srle 32**|\n|165547365| 39.5|**5972**|**10120**|**srle 64**|\n|419225625|100.0|**13938**|**14017**|**memcpy**|\n\n- Text File: [enwik9bwt](http://mattmahoney.net/dc/textdata.html) enwik9 bwt generated w. [libdivsufsort](https://code.google.com/p/libdivsufsort/)\n\n|C Size|ratio%|C MB/s|D MB/s|Name / 2019-08|\n|--------:|-----:|--------:|--------:|----------------|\n|375094084| 37.5|**472**|**1743**|**trle**|\n|415597104| 41.6|461|**3563**|**srle 0**|\n|419263924| 41.9|**568**|**4310**|**srle 8**|\n|487430623| 48.7|**1347**|**6287**|**srle 16**|\n|549202860| 54.9|**2780**|**8238**|**srle 32**|\n|576619941| 57.7|207|     553|   mrle|\n|577685250| 57.8|260|   675|   rle8 1|\n|605759578| 60.6|**5356**|**9471**|**srle 64**|\n|1000000008|100.0|**13931**|**13926**|**memcpy**|\n\n------------------------------------------------------------------------\n#### CPU: ARM A73-ODROID-N2 1.8GHz, gcc 8.3 (2019-08)\n- BMP File: [girl.bmp in RLE64Samples](http://sourceforge.net/projects/nikkhokkho/files/RLE64/3.00/)\n\n|C Size|ratio%|C MB/s|D MB/s|Name / 2019-08|\n|--------:|-----:|--------:|--------:|----------------|\n|2623680|  0.6|**594**|**3399**|**trle**|\n|4148455|  1.0|589|**3799**|**srle 0**|\n|4482384|  1.1|144|3712|mrle|\n|4744806|  1.2|**3365**|3790|**srle 8**|\n|8431844|  2.1|1313|3784|srle 16|\n|13722311|  3.4|1771|**4573**|**srle 32**|\n|19839711|  4.9|**4184**|**6020**|**srle 64**|\n|403920054|100.0|3345|3376|memcpy|\n\n- Text File: [enwik9bwt](http://mattmahoney.net/dc/textdata.html) enwik9 bwt generated w. [libdivsufsort](https://code.google.com/p/libdivsufsort/)\n\n|C Size|ratio%|C MB/s|D MB/s|Name / 2019-08|\n|--------:|-----:|--------:|--------:|----------------|\n|375094084| 37.5|**184**|**722**|**trle**|\n|415597104| 41.6|180|681|srle 0|\n|419263924| 41.9|**244**|693|**srle 8**|\n|487430623| 48.7|**455**|**1203**|**srle 16**|\n|549202860| 54.9|**825**|**2094**|**srle 32**|\n|576619941| 57.7|94|249|mrle|\n|605759578| 60.6|**1677**|**3662**|**srle 64**|\n|1000000008|100.0|**3830**|**3841**|**memcpy**|\n\n------------------------------------------------------------------------\n- Post-processing: Entropy Coding after Run Length Encoding\u003cbr\u003e\n  Direct entropy encoding after \"trle\" (no additional \"move to front\" or other transformation)\n\n|C Size|ratio%|C MB/s|D MB/s|Name /             CPU Skylake 3.4 GHz (2019-06)|\n|--------:|-----:|--------:|--------:|-----------------------------------------------|\n|180510948| 18.1|**154**| **132**|**trle + TurboRC o0** (order 0 bitwise Range Coder)|\n|187099490| 18.7|23|**2560**|**trle + TurboHF 0** (Huffman Coding)|\n|192420471| 19.2|**1527**|**3834**|**trle + TurboANX 12** (Asymmetric Numeral Systems)|\n|193455670| 19.3|**2192**|2986|**trle + TurboHF 12**|\n|197974078| 19.8|1078|1406|trle + fse (Finite State Entropy)|\n|229693376| 23.0|126|106|rle8 + TurboRC o0|\n|254312056| 25.4|119|105|mrle + TurboRC o0|\n\nfor more info, see also: [Entropy Coding Benchmark](https://sites.google.com/site/powturbo/entropy-coder)\n\n### Compile:\n\n  \t\tgit clone https://github.com/powturbo/Turbo-Run-Length-Encoding.git\n        cd Turbo-Run-Length-Encoding\n\n##### Linux + Windows MingW \n \n  \t\tmake\n        or\n  \t\tmake AVX2=1\n\n##### Windows Visual C++\n\n  \t\tnmake /f makefile.vs\n        or\n  \t\tnmake AVX2=1 /f makefile.vs\n\n## Testing Run Length Encoding libraries:\n\n        ./trle file\n        ./trle -e# file\n\n\t\t# = function id (see file trle.c)\n\n### Environment:\n\n###### OS/Compiler (32 + 64 bits):\n- Windows: MinGW-w64 makefile\n- Windows: Visual c++ (\u003e=VS2008) - makefile.vs (for nmake)\n- Windows: Visual Studio project file - vs/vs2017\n- Linux amd64: GNU GCC (\u003e=4.6)\n- Linux amd64: Clang (\u003e=3.2) \n- Linux arm64: 64 bits aarch64 ARMv8:  gcc (\u003e=6.3)\n- Linux arm64: 64 bits aarch64 ARMv8:  clang\n- MaxOS: XCode (\u003e=9)\n- PowerPC ppc64le (incl. SIMD): gcc (\u003e=8.0)\n\n#### Run Length Encoding References:\n  - [Real-Time Compression of IEC 61869-9 Sampled Value Data](https://pure.strath.ac.uk/portal/files/55444712/Blair_etal_AMPS2016_Real_time_compression_of_IEC_61869_9_sampled_value_data.pdf)\n  - [Understanding Compression: Data Compression for Modern Developers](https://books.google.de/books?id=2C2rDAAAQBAJ\u0026pg=PA216\u0026lpg=PA216\u0026dq=%22turborle%22\u0026source=bl\u0026ots=TiLU4Qf47s\u0026sig=tkk0Dnk9NnU0JMR3Z6iW4TRquxg\u0026hl=de\u0026sa=X\u0026ved=0ahUKEwjZq-Li5uXSAhXFCJoKHe77B6cQ6AEIyAEwHQ#v=onepage\u0026q=%22turborle%22\u0026f=false)\n  - [Understanding Compression inl. Run Length Encoding](http://file.allitebooks.com/20160805/Understanding%20Compression.pdf)\n  - [Entropy Coder Benchmark](https://sites.google.com/site/powturbo/entropy-coder)\n\nLast update: 1 APR 2023\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpowturbo%2FTurbo-Run-Length-Encoding","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpowturbo%2FTurbo-Run-Length-Encoding","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpowturbo%2FTurbo-Run-Length-Encoding/lists"}