{"id":13731899,"url":"https://github.com/aras-p/smol-v","last_synced_at":"2025-04-05T15:10:20.733Z","repository":{"id":140930533,"uuid":"66714483","full_name":"aras-p/smol-v","owner":"aras-p","description":"SMOL-V: like Vulkan/Khronos SPIR-V, but smaller.","archived":false,"fork":false,"pushed_at":"2024-09-26T07:27:21.000Z","size":3229,"stargazers_count":331,"open_issues_count":1,"forks_count":14,"subscribers_count":15,"default_branch":"main","last_synced_at":"2025-04-05T00:47:52.692Z","etag":null,"topics":["compression","spir-v","vulkan"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aras-p.png","metadata":{"files":{"readme":"README.md","changelog":"Changelog.md","contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-08-27T12:55:58.000Z","updated_at":"2025-03-25T17:11:48.000Z","dependencies_parsed_at":null,"dependency_job_id":"dc77803d-b355-46ef-99f2-c53cf60c6f8c","html_url":"https://github.com/aras-p/smol-v","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aras-p%2Fsmol-v","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aras-p%2Fsmol-v/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aras-p%2Fsmol-v/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aras-p%2Fsmol-v/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aras-p","download_url":"https://codeload.github.com/aras-p/smol-v/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247353749,"owners_count":20925329,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compression","spir-v","vulkan"],"created_at":"2024-08-03T02:01:40.998Z","updated_at":"2025-04-05T15:10:20.706Z","avatar_url":"https://github.com/aras-p.png","language":"C++","funding_links":[],"categories":["Graphics"],"sub_categories":[],"readme":"# SMOL-V: like Vulkan/Khronos SPIR-V, but smaller. ![Build Status](https://github.com/aras-p/smol-v/workflows/build_and_test/badge.svg)\n\nTL;DR: makes your SPIR-V shaders about 3x smaller, and decodes very fast. If you like to compress them with regular compressor like zlib/lz4/zstd,\nthen smol-v before the compression can still make them 2x smaller.\n\n![](/size_chart.png)\n\n## Overview\n\nSMOL-V encodes Vulkan/Khronos [SPIR-V](https://www.khronos.org/registry/spir-v/)\nformat programs into a form that is *smoller*, and is more\ncompressible. Normally no changes to the programs are done; they decode\ninto exactly same program as what was encoded. Optionally, debug information\ncan be removed too.\n\nSPIR-V is a very verbose format, several times larger than same programs expressed in other\nshader formats *(e.g. DX11 bytecode, GLSL, DX9 bytecode etc.)*. The SSA-form with ever increasing\nIDs is not very appreciated by regular data compressors either. SMOL-V does several things\nto improve this:\n\n- Many words, especially ones that most often have small values, are encoded using\n  [\"varint\" scheme](https://developers.google.com/protocol-buffers/docs/encoding) (1-5 bytes per\n  word, with just one byte for values in 0..127 range).\n- Some IDs used in the program are delta-encoded, relative to previously seen IDs (e.g. Result\n  IDs). Often instructions reference things that were computed just before, so this results in\n  small deltas. These values are also encoded using \"varint\" scheme.\n- Reordering instruction opcodes so that the most common ones are the smallest values, for smaller\n  varint encoding.\n- Encoding several instructions in a more compact form, e.g. the \"typical \u003c=4 component swizzle\"\n  shape of a VectorShuffle instruction, or sequences of MemberDecorate instructions.\n\nA somewhat similar utility is [spirv-remap from glslang](https://github.com/KhronosGroup/glslang/blob/main/README-spirv-remap.txt).\n\nSee [this blog post](https://aras-p.info/blog/2016/09/01/SPIR-V-Compression/) for more information about\nhow I did SMOL-V.\n\n\n## Usage\n\nAdd [`source/smolv.h`](source/smolv.h) and [`source/smolv.cpp`](source/smolv.cpp) to your C++ project build.\nIt might require C++11 or somesuch. There are \n[Github Actions](https://github.com/aras-p/smol-v/actions) set up for this project, that build on Windows (VS2022),\nmacOS (Xcode 15) and Linux (Ubuntu 22 / gcc 11).\n\n`smolv::Encode` and `smolv::Decode` is the basic functionality. See [smolv.h](source/smolv.h).\n\nSPIR-V versions up to and including 1.6 are supported currently.\n\nOther functions are for development/statistics purposes, to figure out frequencies and\ndistributions of the instructions.\n\nThere's a test + compression benchmarking suite in `testing/testmain.cpp`, using that needs adding\nother files under testing/external to the build too (3rd party code: glslang remapper 14.3.0, Zstd 1.5.6, LZ4 1.10, miniz).\n\n## Changelog\n\nSee [**Changelog**](Changelog.md).\n\n\n## Limitations / TODO\n\n- SPIR-V where the words got stored in big-endian layout is not supported yet.\n- The whole thing might not work on Big-Endian CPUs. It might, but I'm not 100% sure.\n- Not much prevention is done against malformed/corrupted inputs, TODO.\n- Out of memory cases are not handled. The code will either throw exception\n  or crash, depending on your compilation flags.\n\n\n## License\n\nCode itself: **Public Domain** or **MIT**, pick whichever works better for you.\n\nThere is 3rd party code under the testing framework (`testing/external`); it is not required for\nusing SMOL-V. Most of that code ([glslang](https://github.com/KhronosGroup/glslang),\n[LZ4](https://github.com/Cyan4973/lz4), [Zstd](https://github.com/facebook/zstd), [sokol_time.h](https://github.com/floooh/sokol/blob/master/sokol_time.h))\nis BSD or zlib-licensed, and taken from github repositories of the respective projects. [miniz](https://github.com/richgel999/miniz)\nis public domain.\n\nThere are SPIR-V binary shader dumps under `tests/spirv-dumps` for compression testing;\nthese are not required for using SMOL-V. Not sure how to appropriately\n\"license\" them (but hey they are kinda useless by themselves out of context),\nso I'll go with this: \"Binary shader dumps under 'tests' folder are only to be\nused for SMOL-V testing\". Details on them:\n\n* `tests/spirv-dumps/dota2` - some shaders from [DOTA2](http://blog.dota2.com/), Copyright Valve Corporation, all rights reserved.\n* `tests/spirv-dumps/shadertoy` - several most popular shaders from [Shadertoy](https://www.shadertoy.com/), converted to Vulkan\n  SPIR-V via glslang. Copyrights by their individual authors (filename matches last component of shadertoy URL).\n* `tests/spirv-dumps/talos` - some shaders from [The Talos Principle](http://www.croteam.com/talosprinciple/),\n  Copyright (c) 2002-2016 Croteam All rights reserved.\n* `tests/spirv-dumps/unity` - various [Unity](https://unity3d.com/) shaders, produced\n  through a HLSL -\u003e DX11 bytecode -\u003e HLSLcc -\u003e glslang toolchain.\n\n\n\n## Results\n\nAs of 2024 Sep 23, results on 383 shaders (under `tests/spirv-dumps`) are:\n\n```\nCompressed with \u003cnone\u003e:\nRaw        5948.7KB 100.0%\nRemapper   5602.9KB  94.2%\nSmolV      2020.9KB  34.0%\nCompressed with zlib:\nRaw        1550.6KB  26.1%\nRemapper   1419.2KB  23.9%\nSmolV       790.6KB  13.3%\nCompressed with LZ4 HC:\nRaw        1769.6KB  29.7%\nRemapper   1551.1KB  26.1%\nSmolV       823.1KB  13.8%\nCompressed with Zstandard:\nRaw        1125.5KB  18.9%\nRemapper    887.4KB  14.9%\nSmolV       573.1KB   9.6%\nCompressed with Zstandard 20:\nRaw         834.5KB  14.0%\nRemapper    666.7KB  11.2%\nSmolV       497.4KB   8.4%\n```\n\nDecoding these 383 shaders from SMOL-V back into SPIR-V takes 10.1ms (VS2022, x64 Release, AMD Ryzen 5950X, one thread).\n\n* \"Raw\" is just raw SPIR-V, with no extra processing.\n* \"Remapper\" is spirv-remap from glslang, with debug info stripping.\n* SmolV is what you're looking at, with debug info stripping too.\n* zlib, LZ4HC and Zstd are general compression algorithms at default settings (Zstd20 is Zstd compression with almost max setting of 20).\n* Compression is done on the whole blob of all the test programs (not individually for each program).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faras-p%2Fsmol-v","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faras-p%2Fsmol-v","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faras-p%2Fsmol-v/lists"}