{"id":13418644,"url":"https://github.com/BinomialLLC/crunch","last_synced_at":"2025-03-15T03:31:48.801Z","repository":{"id":48640515,"uuid":"61267002","full_name":"BinomialLLC/crunch","owner":"BinomialLLC","description":"Advanced DXTc texture compression and transcoding library","archived":false,"fork":false,"pushed_at":"2023-01-17T15:09:57.000Z","size":9321,"stargazers_count":844,"open_issues_count":30,"forks_count":183,"subscribers_count":49,"default_branch":"master","last_synced_at":"2024-07-31T22:43:39.343Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://binomial.info","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BinomialLLC.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-06-16T06:12:08.000Z","updated_at":"2024-07-29T17:18:23.000Z","dependencies_parsed_at":"2023-02-10T10:15:16.836Z","dependency_job_id":null,"html_url":"https://github.com/BinomialLLC/crunch","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BinomialLLC%2Fcrunch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BinomialLLC%2Fcrunch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BinomialLLC%2Fcrunch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BinomialLLC%2Fcrunch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BinomialLLC","download_url":"https://codeload.github.com/BinomialLLC/crunch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243681024,"owners_count":20330152,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T22:01:04.935Z","updated_at":"2025-03-15T03:31:44.164Z","avatar_url":"https://github.com/BinomialLLC.png","language":"C++","readme":"crunch/crnlib v1.04 - Advanced DXTn texture compression library\n\nPublic Domain - Please see license.txt. \n\nPortions of this software make use of public domain code originally\nwritten by Igor Pavlov (LZMA), RYG (crn_ryg_dxt*), and Sean Barrett (stb_image.c).\n\nIf you use this software in a product, an acknowledgment in the product \ndocumentation would be highly appreciated but is not required.\n\n## Overview\n\ncrnlib is a lossy texture compression library for developers that ship\ncontent using the DXT1/5/N or 3DC compressed color/normal map/cubemap\nmipmapped texture formats. It was written by the same author as the open\nsource [LZHAM compression library](http://code.google.com/p/lzham/).\n\nIt can compress mipmapped 2D textures, normal maps, and cubemaps to\napprox. 1-1.25 bits/texel, and normal maps to 1.75-2 bits/texel. The\nactual bitrate depends on the complexity of the texture itself, the\nspecified quality factor/target bitrate, and ultimately on the desired\nquality needed for a particular texture. \n\ncrnlib's differs significantly from other approaches because its\ncompressed texture data format was carefully designed to be quickly\ntranscodable directly to DXTn with no intermediate recompression step.\nThe typical (single threaded) transcode to DXTn rate is generally\nbetween 100-250 megatexels/sec. The current library supports PC\n(Win32/x64) and Xbox 360. Fast random access to individual mipmap levels\nis supported.\n\ncrnlib can also generates standard .DDS files at specified quality\nsetting, which results in files that are much more compressible by\nLZMA/Deflate/etc. compared to files generated by standard DXTn texture\ntools (see below). This feature allows easy integration into any engine\nor graphics library that already supports .DDS files.\n\nThe .CRN file format supports the following core DXTn texture formats:\nDXT1 (but not DXT1A), DXT5, DXT5A, and DXN/3DC\n\nIt also supports several popular swizzled variants (several are\nalso supported by AMD's Compressonator): \nDXT5_XGBR, DXT5_xGxR, DXT5_AGBR, and DXT5_CCxY (experimental luma-chroma YCoCg).\n\n## Recommended Software\n\nAMD's [Compressonator tool](https://github.com/GPUOpen-Tools/Compressonator)\nis recommended to view the .DDS files created by the crunch tool and the included example projects.\n\nNote: Some of the swizzled DXTn .DDS output formats (such as DXT5_xGBR)\nread/written by the crunch tool or examples deviate from the DX9 DDS\nstandard, so DXSDK tools such as DXTEX.EXE won't load them at all or\nthey won't be properly displayed.\n\n## Compression Algorithm Details\n\nThe compression process employed in creating both .CRN and\nclustered .DDS files utilizes a very high quality, scalable DXTn\nendpoint optimizer capable of processing any number of pixels (instead\nof the typical hard coded 16), optional adaptive switching between\nseveral macroblock sizes/configurations (currently any combination of\n4x4, 8x4, 4x8, and 8x8 pixel blocks), endpoint clusterization using\ntop-down cluster analysis, vector quantization (VQ) of the selector\nindices, and several custom algorithms for compressing the resulting\nendpoint/selector codebooks and macroblock indices. Multiple feedback\npasses are performed between the clusterization and VQ steps to optimize\nquality, and several steps use a brute force refinement approach to improve \nquality. The majority of compression steps are multithreaded.\n\nThe .CRN format currently utilizes canonical Huffman coding for speed\n(similar to Deflate but with much larger tables), but the next major\nversion will also utilize adaptive binary arithmetic coding and higher\norder context modeling using already developed tech from the my LZHAM\ncompression library.\n\n## Supported File Formats\n\ncrnlib supports two compressed texture file formats. The first\nformat (clustered .DDS) is simple to integrate into an existing project\n(typically, no code changes are required), but it doesn't offer the\nhighest quality/compression ratio that crnlib is capable of. Integrating\nthe second, higher quality custom format (.CRN) requires a few\ntypically straightforward engine modifications to integrate the\n.CRN-\u003eDXTn transcoder header file library into your tools/engine.\n\n### .DDS\n\ncrnlib can compress textures to standard DX9-style .DDS files using\nclustered DXTn compression, which is a subset of the approach used to\ncreate .CRN files.(For completeness, crnlib also supports vanilla, block\nby block DXTn compression too, but that's not very interesting.)\nClustered DXTn compressed .DDS files are much more compressible than\nfiles created by other libraries/tools. Apart from increased\ncompressibility, the .DDS files generated by this process are completely\nstandard so they should be fairly easy to add to a project with little\nto no code changes.\n\nTo actually benefit from clustered DXTn .DDS files, your engine needs to\nfurther losslessly compress the .DDS data generated by crnlib using a\nlossless codec such as zlib, lzo, LZMA, LZHAM, etc. Most likely, your\nengine does this already. (If not, you definitely should because DXTn\ncompressed textures generally contain a large amount of highly redundant\ndata.)\n\nClustered .DDS files are intended to be the simplest/fastest way to\nintegrate crnlib's tech into a project.\n\n### .CRN\n\nThe second, better, option is to compress your textures to .CRN files\nusing crnlib. To read the resulting .CRN data, you must add the .CRN\ntranscoder library (located in the included single file, stand-alone\nheader file library inc/crn_decomp.h) into your application. .CRN files\nprovide noticeably higher quality at the same effective bitrate compared\nto clustered DXTn compressed .DDS files. Also, .CRN files don't require\nfurther lossless compression because they're already highly compressed.\n\n.CRN files are a bit more difficult/risky to integrate into a project, but\nthe resulting compression ratio and quality is superior vs. clustered .DDS files.\n\n### .KTX\n\ncrnlib and crunch can read/write the .KTX file format in various pixel formats.\nRate distortion optimization (clustered DXTc compression) is not yet supported\nwhen writing .KTX files. \n\nThe .KTX file format is just like .DDS, except it's a fairly well specified\nstandard created by the Khronos Group. Unfortunately, almost all of the tools I've\nfound that support .KTX are fairly (to very) buggy, or are limited to only a handful\nof pixel formats, so there's no guarantee that the .KTX files written by crnlib can\nbe reliably read by other tools.\n\n## Building the Examples\n\nThis release contains the source code and projects for three simple\nexample projects:\n\ncrn_examples.2008.sln is a Visual Studio 2008 (VC9) solution file\ncontaining projects for Win32 and x64. crnlib itself also builds with\nVS2005, VS2010, and gcc 4.5.0 (TDM GCC+MinGW).  A codeblocks 10.05\nworkspace and project file is also included, but compiling crnlib this\nway hasn't been tested much.\n\n### example1\n\nDemonstrates how to use crnlib's high-level C-helper\ncompression/decompression/transcoding functions in inc/crnlib.h. It's a\nfairly complete example of crnlib's functionality.\n\n### example2\nShows how to transcodec .CRN files to .DDS using **only**\nthe functionality in inc/crn_decomp.h. It does not link against against\ncrnlib.lib or depend on it in any way. (Note: The complete source code,\napprox. 4800 lines, to the CRN transcoder is included in inc/crn_decomp.h.)\n\nexample2 is intended to show how simple it is to integrate CRN textures\ninto your application.\n\n### example3\nShows how to use the regular, low-level DXTn block compressor\nfunctions in inc/crnlib.h. This functionality is included for\ncompleteness. (Your engine or toolchain most likely already has its own\nDXTn compressor. crnlib's compressor is typically very competitive or\nsuperior to most available closed and open source CPU-based\ncompressors.)\n\n## Creating Compressed Textures from the Command Line (crunch.exe)\n\nThe simplest way to create compressed textures using crnlib is to\nintegrate the bin\\crunch.exe or bin\\crunch_x64.exe) command line tool\ninto your texture build toolchain or export process. It can write DXTn\ncompressed 2D/cubemap textures to regular DXTn compressed .DDS,\nclustered (or reduced entropy) DXTn compressed .DDS, or .CRN files. It\ncan also transcode or decompress files to several standard image\nformats, such as TGA or BMP. Run crunch.exe with no options for help.\n\nThe .CRN files created by crunch.exe can be efficiently transcoded to\nDXTn using the included CRN transcoding library, located in full source\nform under inc/crn_decomp.h.\n\nHere are a few example crunch.exe command lines:\n\n1. Compress blah.tga to blah.dds using normal DXT1 compression:\n  * `crunch -file blah.tga -fileformat dds -dxt1`\n\n2. Compress blah.tga to blah.dds using clustered DXT1 at an effective bitrate of 1.5 bits/texel, display image statistic:\n  * `crunch -file blah.tga -fileformat dds -dxt1 -bitrate 1.5 -imagestats`\n\n3. Compress blah.tga to blah.dds using clustered DXT1 at quality level 100 (from [0,255]), with no mipmaps, display LZMA statistics:\n  * `crunch -file blah.tga -fileformat dds -dxt1 -quality 100 -mipmode none -lzmastats`\n\n3. Compress blah.tga to blah.crn using clustered DXT1 at a bitrate of 1.2 bits/texel, no mipmaps:\n  * `crunch -file blah.tga -dxt1 -bitrate 1.2 -mipmode none`\n\n4. Decompress blah.dds to a .tga file:\n  * `crunch -file blah.dds -fileformat tga`\n\n5. Transcode blah.crn to a .dds file:\n  * `crunch -file blah.crn`\n\n6. Decompress blah.crn, writing each mipmap level to a separate .tga file:\n  * `crunch -split -file blah.crn -fileformat tga`\n\ncrunch.exe can do a lot more, like rescale/crop images before\ncompression, convert images from one file format to another, compare\nimages, process multiple images, etc.\n\nNote: I would have included the full source to crunch.exe, but it still\nhas some low-level dependencies to crnlib internals which I didn't have\ntime to address. This version of crunch.exe has some reduced\nfunctionality compared to an earlier eval release. For example, XML file\nsupport is not included in this version.\n\n## Using crnlib\n\nThe most flexible and powerful way of using crnlib is to integrate the\nlibrary into your editor/toolchain/etc. and directly supply it your\nraw/source texture bits. See the C-style API's and comments in\ninc/crnlib.h.\n\nTo compress, you basically fill in a few structs in and call one function:\n\n```c\nvoid *crn_compress( const crn_comp_params \u0026comp_params,\n                    crn_uint32 \u0026compressed_size,\n                    crn_uint32 *pActual_quality_level = NULL,\n                    float *pActual_bitrate = NULL);\n```\n\nOr, if you want crnlib to also generate mipmaps, you call this function:\n\n```c\nvoid *crn_compress( const crn_comp_params \u0026comp_params,\n                    const crn_mipmap_params \u0026mip_params,\n                    crn_uint32 \u0026compressed_size,\n                    crn_uint32 *pActual_quality_level = NULL,\n                    float *pActual_bitrate = NULL);\n```\n\nYou can also transcode/uncompress .DDS/.CRN files to raw 32bpp images\nusing `crn_decompress_crn_to_dds()` and `crn_decompress_dds_to_images()`.\n\nInternally, crnlib just uses inc/crn_decomp.h to transcode textures to\nDXTn. If you only need to transcode .CRN format files to raw DXTn bits\nat runtime (and not compress), you don't actually need to compile or\nlink against crnlib at all. Just include inc/crn_decomp.h, which\ncontains a completely self-contained CRN transcoder in the \"crnd\"\nnamespace. The `crnd_get_texture_info()`, `crnd_unpack_begin()`,\n`crnd_unpack_level()`, etc. functions are all you need to efficiently get\nat the raw DXTn bits, which can be directly supplied to whatever API or\nGPU you're using. (See example2.)\n\nImportant note: When compiling under native client, be sure to define\nthe `PLATFORM_NACL` macro before including the `inc/crn_decomp.h` header file library.\n\n## Known Issues/Bugs\n\n* crnlib currently assumes you'll be further losslessly compressing its\noutput .DDS files using LZMA. However, some engines use weaker codecs\nsuch as LZO, zlib, or custom codecs, so crnlib's bitrate measurements\nwill be inaccurate. It should be easy to allow the caller to plug-in\ncustom lossless compressors for bitrate measurement.\n\n* Compressing to a desired bitrate can be time consuming, especially when\nprocessing large (2k or 4k) images to the .CRN format. There are several\nhigh-level optimizations employed when compressing to clustered DXTn .DDS\nfiles using multiple trials, but not so for .CRN.\n\n* The .CRN compressor does not currently use 3 color (transparent) DXT1\nblocks at all, only 4 color blocks. So it doesn't support DXT1A\ntransparency, and its output quality suffers a little due to this\nlimitation. (Note that the clustered DXTn compressor used when\nwriting clustered .DDS files does *not* have this limitation.)\n\n* Clustered DXT5/DXT5A compressor is able to group DXT5A blocks into\nclusters only if they use absolute (black/white) selector indices. This\nhurts performance at very low bitrates, because too many bits are\neffectively given to alpha.\n\n* DXT3 is not supported when writing .CRN or clustered DXTn DDS files.\n(DXT3 is supported by crnlib's when compressing to regular DXTn DDS\nfiles.) You'll get DXT5 files if you request DXT3. However, DXT3 is\nsupported by the regular DXTn block compressor. (DXT3's 4bpp fixed alpha\nsucks verses DXT5 alpha blocks, so I don't see this as a bug deal.)\n\n* The DXT5_CCXY format uses a simple YCoCg encoding that is workable but\nhasn't been tuned for max. quality yet.\n\n* Clustered (or rate distortion optimized) DXTc compression is only\nsupported when writing to .DDS, not .KTX. Also, only plain block by block\ncompression is supported when writing to ETC1, and .CRN does not support ETC1.\n\n## Compile to Javascript with Emscripten\n\nDownload and install Emscripten:\n    http://kripken.github.io/emscripten-site/docs/getting_started/downloads.html\n\nFrom the root directory, run:\n```c\n    emcc -O3 emscripten/crn.cpp -I./inc -s EXPORTED_FUNCTIONS=\"['_malloc', '_free', '_crn_get_width', '_crn_get_height', '_crn_get_levels', '_crn_get_dxt_format', '_crn_get_bytes_per_block', '_crn_get_uncompressed_size', '_crn_decompress']\" -s NO_EXIT_RUNTIME=1 -s NO_FILESYSTEM=1 -s ELIMINATE_DUPLICATE_FUNCTIONS=1 -s ALLOW_MEMORY_GROWTH=1 --memory-init-file 0 -o crunch.js\n```\n","funding_links":[],"categories":["TODO scan for Android support in followings","C++","Graphics"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBinomialLLC%2Fcrunch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBinomialLLC%2Fcrunch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBinomialLLC%2Fcrunch/lists"}