{"id":15664488,"url":"https://github.com/ybubnov/libhuffman","last_synced_at":"2025-05-05T23:30:01.923Z","repository":{"id":14253728,"uuid":"16961243","full_name":"ybubnov/libhuffman","owner":"ybubnov","description":"The Huffman library is a simple, pure C99 library for encoding and decoding data using a frequency-sorted binary tree.","archived":false,"fork":false,"pushed_at":"2024-02-24T20:35:08.000Z","size":187,"stargazers_count":18,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-01T18:11:50.464Z","etag":null,"topics":["c","compression","decompression","huffman","python"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ybubnov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-02-18T19:42:23.000Z","updated_at":"2025-02-15T16:08:34.000Z","dependencies_parsed_at":"2024-10-03T13:42:55.131Z","dependency_job_id":"3b48e04e-5746-4119-b2c7-36fafa3c8421","html_url":"https://github.com/ybubnov/libhuffman","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ybubnov%2Flibhuffman","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ybubnov%2Flibhuffman/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ybubnov%2Flibhuffman/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ybubnov%2Flibhuffman/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ybubnov","download_url":"https://codeload.github.com/ybubnov/libhuffman/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252592675,"owners_count":21773314,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","compression","decompression","huffman","python"],"created_at":"2024-10-03T13:42:47.500Z","updated_at":"2025-05-05T23:30:01.906Z","avatar_url":"https://github.com/ybubnov.png","language":"C","readme":"# libhuffman - The Huffman coding library\n\nThe Huffman library is a simple, pure C library for encoding and decoding data using a\nfrequency-sorted binary tree.\nThe implementation of this library is pretty straightforward, additional information\nregarding Huffman coding could be gained from the [Wikipedia](https://en.wikipedia.org/wiki/Huffman_coding).\n\n## Installation\n\nThe build mechanism of the library is based on the [CMake](https://cmake.org) tool, so\nyou could easily install it on your distribution in the following way:\n```bash\n$ sudo cmake install\n```\n\nBy default the command above install the library into `/usr/local/lib` and all\nrequired headers into `/usr/local/include`. The installation process is organized\nusing CMake. Just create a new directory `build` and generate required makefiles:\n```bash\n$ mkdir -p build\n$ cmake ..\n```\n\nAfter that run the `install` target:\n```bash\n$ make install\n```\n\n## Usage\n\n### Encoding\n\nTo encode the data, use either a file stream `huf_fdopen` or `huf_memopen` to use\nan in-memory stream. Consider the following example, where the input is a memory\nstream and output of the encoder is also memory buffer of 1MiB size.\n```c\nvoid *bufin, *bufout = NULL;\nhuf_read_writer_t *input, *output = NULL;\n\n// Allocate the necessary memory.\nhuf_memopen(\u0026input, \u0026bufin, HUF_1MIB_BUFFER);\nhuf_memopen(\u0026output, \u0026bufout, HUF_1MIB_BUFFER);\n\n// Write the data for encoding to the input.\nsize_t input_len = 10;\ninput-\u003ewrite(input-\u003estream, \"0123456789\", input_len);\n```\n\nCreate a configuration used to encode the input string using Huffman algorithm:\n```c\nhuf_config_t config = {\n   .reader = input,\n   .length = input_len,\n   .writer = output,\n   .blocksize = HUF_64KIB_BUFFER,\n};\n\nhuf_error_t err = huf_encode(\u0026config);\nprintf(\"%s\\n\", huf_error_string(err));\n```\n\n- `reader` - input ready for the encoding.\n- `writer` - output for the encoded data.\n- `length` - length of the data in bytes to encode.\n- `blocksize` - the length of each chunk in bytes (instead of reading the file twice\n`libhuffman` reads and encodes data by blocks).\n- `reader_buffer_size` - this is opaque reader buffer size in bytes, if the buffer size\nis set to zero, all reads will be unbuffered.\n- `writer_buffer_size` - this is opaque writer buffer size ib bytes, if the buffer size\nis set to zero, all writes will be unbuffered.\n\nAfter the encoding, the output memory buffer could be automatically scaled to fit all\nnecessary encoded bytes. To retrieve a new length of the buffer, use the following:\n```c\nsize_t out_size = 0;\nhuf_memlen(output, \u0026out_size);\n\n// The data is accessible through the `bufout` variable or using `read` function:\nuint8_t result[10] = {0};\nsize_t result_len = 10;\n\n// result_len is inout parameter, and will contain the length of encoding\n// after the reading from the stream.\noutput-\u003eread(output-\u003estream, result, \u0026result_len);\n```\n\n### Decoding\n\nDecoding is similar to the encoding, except that reader attribute of the configuration\nshould contain the data used to decode:\n```c\ninput-\u003ewrite(input-\u003estream, decoding, decoding_len);\n\nhuf_config_t config = {\n    .reader = input,\n    .length = input_len,\n    .writer = output,\n    .blockize = HUF_64KIB_BUFFER,\n};\n\n\n// After the decoding the original data will be writter to the `output`.\nhuf_decode(\u0026config);\n```\n\n### Resource Deallocation\n\nOnce the processing of the encoding is completed, consider freeing the allocated memory:\n```c\n// This does not free underlying buffer, call free for the buffer.\nhuf_memclose(\u0026mem_out);\n\nfree(buf);\n```\n\nFor more examples, please, refer to the [`tests`](tests) directory.\n\n## Python Bindings\n\nPython bindings for `libhuffman` library are distributed as PyPI package, to install\nthat package, execute the following command:\n```sh\npip install huffmanfile\n```\n\nYou can use the `libhuffman` for performant compression and decompression of Huffman\nencoding. The interface of the Python library is similar to the interface of the\n[`bz2`](https://docs.python.org/3/library/bz2.html) and\n[`lzma`](https://docs.python.org/3/library/lzma.html) packages from Python's standard\nlibrary.\n\n### Examples of usage\n\nReading in a compressed file:\n```py\nimport huffmanfile\nwith huffmanfile.open(\"file.hm\") as f:\n    file_content = f.read()\n```\n\nCreating a compressed file:\n```py\nimport huffmanfile\ndata = b\"Insert Data Here\"\nwith huffmanfile.open(\"file.hm\", \"w\") as f:\n    f.write(data)\n```\n\nCompressing data in memory:\n```py\nimport huffmanfile\ndata_in = b\"Insert Data Here\"\ndata_out = huffmanfile.compress(data_in)\n```\n\nIncremental compression:\n```py\nimport huffmanfile\nhfc = huffmanfile.HuffmanCompressor()\nout1 = hfc.compress(b\"Some data\\n\")\nout2 = hfc.compress(b\"Another piece of data\\n\")\nout3 = hfc.compress(b\"Even more data\\n\")\nout4 = hfc.flush()\n# Concatenate all the partial results:\nresult = b\"\".join([out1, out2, out3, out4])\n```\n\nNote, random data tends to compress poorly, while ordered, repetitive data usually\nyields a high compression ratio.\n\n## License\n\nThe Huffman library is distributed under MIT license, therefore you are free to do with\ncode whatever you want. See the [LICENSE](LICENSE) file for full license text.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fybubnov%2Flibhuffman","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fybubnov%2Flibhuffman","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fybubnov%2Flibhuffman/lists"}