{"id":20964549,"url":"https://github.com/hyparam/hysnappy","last_synced_at":"2025-05-14T09:32:46.268Z","repository":{"id":223067356,"uuid":"754480614","full_name":"hyparam/hysnappy","owner":"hyparam","description":"Snappy decompression with WebAssembly","archived":false,"fork":false,"pushed_at":"2024-04-26T21:17:14.000Z","size":218,"stargazers_count":9,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-08-10T11:05:14.915Z","etag":null,"topics":["compression","parquet","snappy","wasm","webassembly"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hyparam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-02-08T06:18:51.000Z","updated_at":"2024-06-30T16:23:20.000Z","dependencies_parsed_at":"2024-02-18T03:27:18.052Z","dependency_job_id":"2177653a-2ddc-465c-a4c2-605e90440166","html_url":"https://github.com/hyparam/hysnappy","commit_stats":null,"previous_names":["hyparam/hysnappy"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyparam%2Fhysnappy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyparam%2Fhysnappy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyparam%2Fhysnappy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyparam%2Fhysnappy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hyparam","download_url":"https://codeload.github.com/hyparam/hysnappy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225285535,"owners_count":17450057,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compression","parquet","snappy","wasm","webassembly"],"created_at":"2024-11-19T02:56:01.887Z","updated_at":"2024-11-19T02:56:02.446Z","avatar_url":"https://github.com/hyparam.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HySnappy\n\n![hysnappy penguin](hysnappy.jpg)\n\n[![npm](https://img.shields.io/npm/v/hysnappy)](https://www.npmjs.com/package/hysnappy)\n[![workflow status](https://github.com/hyparam/hysnappy/actions/workflows/ci.yml/badge.svg)](https://github.com/hyparam/hysnappy/actions)\n[![mit license](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n[![dependencies](https://img.shields.io/badge/Dependencies-0-blueviolet)](https://www.npmjs.com/package/hysnappy?activeTab=dependencies)\n\nSnappy decompression with WebAssembly.\n\nA fast, minimal snappy decompression implementation in C built for WASM.\n\nSnappy compression was released by Google in 2011 with the goal of very high speeds and reasonable compression.\nSnappy is used in various applications.\nFor example, snappy is the default compression format for [Apache Parquet](https://parquet.apache.org) files.\n\n## Usage\n\nThe `snappyUncompress` function expects as arguments: a typed array `compressed`, and an `outputLength` parameter.\nThe length is needed to know how much wasm memory to allocate.\nFor formats like parquet, this length will generally be known in advance.\nTo decompress a `Uint8Array` with known output length:\n\n```js\nimport { snappyUncompress } from 'hysnappy'\n\nconst compressed = new Uint8Array([\n  0x0a, 0x24, 0x68, 0x79, 0x70, 0x65, 0x72, 0x70, 0x61, 0x72, 0x61, 0x6d\n])\nconst outputLength = 10\nconst output = snappyUncompress(compressed, outputLength)\n```\n\n## Hyparquet\n\nHysnappy was built specifically to accelerate the the [hyparquet](https://github.com/hyparam/hyparquet) parquet parsing library.\n\nHysnappy exports a loader function `snappyUncompressor()` which loads the WASM module once, and returns a pre-loaded version of `snappyUncompress` function.\n\nTo use hysnappy with hyparquet:\n\n```js\nimport { parquetRead } from 'hyparquet'\nimport { snappyUncompressor } from 'hysnappy'\n\nparquetRead({ file, compressors: {\n  SNAPPY: snappyUncompressor(),\n}})\n```\n\n## Development\n\nThe build uses clang _without_ emscripten, in order to produce the smallest possible binary.\n\nRun `make` to build from source. The build process consists of:\n\n1. Compile from `snappy.c` to `hysnappy.wasm` using `clang`.\n2. Encode `hysnappy.wasm` as base64 to `hysnappy.wasm.base64`.\n3. Insert base64 string into `hysnappy.js` for distribution.\n\n## WASM Loading\n\nBy keeping `hysnappy.wasm` under 4kb, we can include it directly in the `hysnappy.js` file and load the WASM blob synchronously, which is faster than loading a separate `.wasm` file. [[web.dev]](https://web.dev/articles/loading-wasm)\n\n## References\n\n - https://en.wikipedia.org/wiki/Snappy_(compression)\n - https://github.com/andikleen/snappy-c\n - https://github.com/google/snappy\n - https://github.com/zhipeng-jia/snappyjs\n - https://web.dev/articles/loading-wasm\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyparam%2Fhysnappy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhyparam%2Fhysnappy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyparam%2Fhysnappy/lists"}