{"id":19745291,"url":"https://github.com/blosc/c-blosc","last_synced_at":"2025-05-14T04:07:30.537Z","repository":{"id":1565687,"uuid":"1990838","full_name":"Blosc/c-blosc","owner":"Blosc","description":"A blocking, shuffling and loss-less compression library that can be faster than `memcpy()`.","archived":false,"fork":false,"pushed_at":"2025-03-28T06:57:20.000Z","size":9092,"stargazers_count":1006,"open_issues_count":35,"forks_count":159,"subscribers_count":49,"default_branch":"main","last_synced_at":"2025-05-08T13:36:35.210Z","etag":null,"topics":["c","compression","fast"],"latest_commit_sha":null,"homepage":"https://www.blosc.org","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Blosc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":"FUNDING.yml","license":"LICENSE.txt","code_of_conduct":"code_of_conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["numfocus"],"custom":["https://numfocus.org/donate-to-blosc"]}},"created_at":"2011-07-03T11:38:39.000Z","updated_at":"2025-05-07T10:16:24.000Z","dependencies_parsed_at":"2024-03-15T02:22:06.424Z","dependency_job_id":"89079b2f-9eee-45f0-9314-d77f734d14e9","html_url":"https://github.com/Blosc/c-blosc","commit_stats":{"total_commits":1399,"total_committers":67,"mean_commits":"20.880597014925375","dds":0.4045746962115797,"last_synced_commit":"dcf6813d46f9fcbf7af4395b5536f886dfc76dd0"},"previous_names":[],"tags_count":80,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Blosc%2Fc-blosc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Blosc%2Fc-blosc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Blosc%2Fc-blosc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Blosc%2Fc-blosc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Blosc","download_url":"https://codeload.github.com/Blosc/c-blosc/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254069489,"owners_count":22009557,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","compression","fast"],"created_at":"2024-11-12T02:06:20.261Z","updated_at":"2025-05-14T04:07:30.510Z","avatar_url":"https://github.com/Blosc.png","language":"C","readme":"# Blosc: A blocking, shuffling and lossless compression library\n| Author | Contact | URL |\n|--------|---------|-----|\n| Blosc Development Team | blosc@blosc.org | https://www.blosc.org | \n\n| Gitter | GH Actions | NumFOCUS | Code of Conduct |\n|--------|------------|----------|-----------------|\n| [![Gitter](https://badges.gitter.im/Blosc/c-blosc.svg)](https://gitter.im/Blosc/c-blosc?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge) | [![CI CMake](https://github.com/Blosc/c-blosc/workflows/CI%20CMake/badge.svg)](https://github.com/Blosc/c-blosc/actions?query=workflow%3A%22CI+CMake%22) | [![Powered by NumFOCUS](https://img.shields.io/badge/powered%20by-NumFOCUS-orange.svg?style=flat\u0026colorA=E1523D\u0026colorB=007D8A)](https://numfocus.org) | [![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg)](code_of_conduct.md) |\n\n## What is it?\n\n**Note**: There is a more modern version of this package called C-Blosc2\nwhich supports many more features and is more actively maintained.  Visit it at:\nhttps://github.com/Blosc/c-blosc2\n\nBlosc is a high performance compressor optimized for binary data.\nIt has been designed to transmit data to the processor cache faster\nthan the traditional, non-compressed, direct memory fetch approach via\na memcpy() OS call.  Blosc is the first compressor (that I'm aware of)\nthat is meant not only to reduce the size of large datasets on-disk or\nin-memory, but also to accelerate memory-bound computations.\n\nIt uses the [blocking technique](https://www.blosc.org/docs/StarvingCPUs-CISE-2010.pdf)\nso as to reduce activity in the memory bus as much as possible. In short, this\ntechnique works by dividing datasets in blocks that are small enough\nto fit in caches of modern processors and perform compression /\ndecompression there.  It also leverages, if available, SIMD\ninstructions (SSE2, AVX2) and multi-threading capabilities of CPUs, in\norder to accelerate the compression / decompression process to a\nmaximum.\n\nSee some [benchmarks](https://www.blosc.org/pages/synthetic-benchmarks/) about Blosc performance.\n\nBlosc is distributed using the BSD license, see LICENSE.txt for\ndetails.\n\n## Meta-compression and other differences over existing compressors\n\nC-Blosc is not like other compressors: it should rather be called a\nmeta-compressor.  This is so because it can use different compressors\nand filters (programs that generally improve compression ratio).  At\nany rate, it can also be called a compressor because it happens that\nit already comes with several compressor and filters, so it can\nactually work like a regular codec.\n\nCurrently C-Blosc comes with support of BloscLZ, a compressor heavily\nbased on FastLZ (https://ariya.github.io/FastLZ/), LZ4 and LZ4HC\n(https://lz4.org/), Snappy\n(https://google.github.io/snappy/), Zlib (https://zlib.net/) and\nZstandard (https://facebook.github.io/zstd/).\n\nC-Blosc also comes with highly optimized (they can use\nSSE2 or AVX2 instructions, if available) shuffle and bitshuffle filters\n(for info on how and why shuffling works [see here](https://www.slideshare.net/PyData/blosc-py-data-2014/17?src=clipshare)).\nHowever, additional compressors or filters may be added in the future.\n\nBlosc is in charge of coordinating the different compressor and\nfilters so that they can leverage the \n[blocking technique](https://www.blosc.org/docs/StarvingCPUs-CISE-2010.pdf)\nas well as multi-threaded execution (if several cores are\navailable) automatically. That makes that every codec and filter\nwill work at very high speeds, even if it was not initially designed\nfor doing blocking or multi-threading.\n\nFinally, C-Blosc is specially suited to deal with binary data because\nit can take advantage of the type size meta-information for improved\ncompression ratio by using the integrated shuffle and bitshuffle filters.\n\nWhen taken together, all these features set Blosc apart from other\ncompression libraries.\n\n## Compiling the Blosc library\n\nBlosc can be built, tested and installed using CMake_.\nThe following procedure describes the \"out of source\" build.\n\n```console\n\n  $ cd c-blosc\n  $ mkdir build\n  $ cd build\n```\n\nNow run CMake configuration and optionally specify the installation\ndirectory (e.g. '/usr' or '/usr/local'):\n\n```console\n\n  $ cmake -DCMAKE_INSTALL_PREFIX=your_install_prefix_directory ..\n```\n\nCMake allows to configure Blosc in many different ways, like preferring\ninternal or external sources for compressors or enabling/disabling\nthem.  Please note that configuration can also be performed using UI\ntools provided by [CMake](https://cmake.org) (ccmake or cmake-gui):\n\n```console\n\n  $ ccmake ..      # run a curses-based interface\n  $ cmake-gui ..   # run a graphical interface\n```\n\nBuild, test and install Blosc:\n\n\n```console\n\n  $ cmake --build .\n  $ ctest\n  $ cmake --build . --target install\n```\n\nThe static and dynamic version of the Blosc library, together with\nheader files, will be installed into the specified\nCMAKE_INSTALL_PREFIX.\n\n### Codec support with CMake\n\nC-Blosc comes with full sources for LZ4, LZ4HC, Snappy, Zlib and Zstd\nand in general, you should not worry about not having (or CMake\nnot finding) the libraries in your system because by default the\nincluded sources will be automatically compiled and included in the\nC-Blosc library. This effectively means that you can be confident in\nhaving a complete support for all the codecs in all the Blosc deployments\n(unless you are explicitly excluding support for some of them).\n\nBut in case you want to force Blosc to use external codec libraries instead of\nthe included sources, you can do that:\n\n``` console\n\n  $ cmake -DPREFER_EXTERNAL_ZSTD=ON ..\n```\n\nYou can also disable support for some compression libraries:\n\n\n```console\n\n  $ cmake -DDEACTIVATE_SNAPPY=ON ..  # in case you don't have a C++ compiler\n```\n \n## Examples\n\nIn the [examples/ directory](https://github.com/Blosc/c-blosc/tree/master/examples)\nyou can find hints on how to use Blosc inside your app.\n\n## Supported platforms\n\nBlosc is meant to support all platforms where a C89 compliant C\ncompiler can be found.  The ones that are mostly tested are Intel\n(Linux, Mac OSX and Windows) and ARM (Linux), but exotic ones as IBM\nBlue Gene Q embedded \"A2\" processor are reported to work too.\n\n### Mac OSX troubleshooting\n\nIf you run into compilation troubles when using Mac OSX, please make\nsure that you have installed the command line developer tools.  You\ncan always install them with:\n\n```console\n\n  $ xcode-select --install\n```\n\n## Wrapper for Python\n\nBlosc has an official wrapper for Python.  See:\n\nhttps://github.com/Blosc/python-blosc\n\n## Command line interface and serialization format for Blosc\n\nBlosc can be used from command line by using Bloscpack.  See:\n\nhttps://github.com/Blosc/bloscpack\n\n## Filter for HDF5\n\nFor those who want to use Blosc as a filter in the HDF5 library,\nthere is a sample implementation in the hdf5-blosc project in:\n\nhttps://github.com/Blosc/hdf5-blosc\n\n## Mailing list\n\nThere is an official mailing list for Blosc at:\n\nblosc@googlegroups.com\nhttps://groups.google.com/g/blosc\n\n## Acknowledgments\n\nSee THANKS.rst.\n\n\n----\n\n  **Enjoy data!**\n","funding_links":["https://github.com/sponsors/numfocus","https://numfocus.org/donate-to-blosc"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblosc%2Fc-blosc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fblosc%2Fc-blosc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblosc%2Fc-blosc/lists"}