{"id":40316679,"url":"https://github.com/haskell-text/diablo","last_synced_at":"2026-01-20T07:02:54.224Z","repository":{"id":38453796,"uuid":"422442508","full_name":"haskell-text/diablo","owner":"haskell-text","description":"We sink to the low level, so you don't have to.","archived":false,"fork":false,"pushed_at":"2022-06-01T13:14:44.000Z","size":52,"stargazers_count":3,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2023-03-07T00:05:57.369Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/haskell-text.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-29T04:34:05.000Z","updated_at":"2022-06-01T13:15:12.000Z","dependencies_parsed_at":"2022-08-19T02:30:24.983Z","dependency_job_id":null,"html_url":"https://github.com/haskell-text/diablo","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"purl":"pkg:github/haskell-text/diablo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haskell-text%2Fdiablo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haskell-text%2Fdiablo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haskell-text%2Fdiablo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haskell-text%2Fdiablo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/haskell-text","download_url":"https://codeload.github.com/haskell-text/diablo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haskell-text%2Fdiablo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28597985,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T02:08:49.799Z","status":"ssl_error","status_checked_at":"2026-01-20T02:08:44.148Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-20T07:02:46.019Z","updated_at":"2026-01-20T07:02:54.201Z","avatar_url":"https://github.com/haskell-text.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# diablo\n\n## What is this?\n\nA collection of low-level, byte-array-oriented operations, designed to provide\nefficient implementations for common operations in Haskell programs.\n\n## Wait, it's not a video game?\n\nNo. I just thought `-ldiablo` was funny.\n\n## Why is this needed?\n\nSurely GHC is optimal enough? Well, no:\n\n* GHC, as current, cannot consistently emit SIMD instructions. The only way to\n  do this involves using the LLVM backend, and the optimizer can miss many\n  opportunities, assuming your compiler of choice doesn't ruin your day.\n* Auto-vectorization, even at the level of C, with all the help it can get, is\n  unreliable. Even SIMD _intrinsics_ are \n  [not reliable](https://danluu.com/assembly-intrinsics). When it comes to\n  Haskell, with all of its added complexities, this is a hard problem even in\n  theory.\n* Byte and bit bashing is hard, error-prone and highly counter-intuitive to the\n  untrained. A lot of stuff is lacking implementations, which forces anyone in\n  Haskell to write them _again_ themselves. And this is before contending with\n  the kind of performance surprises GHC Haskell is known for!\n\nThis necessitates a _portable_, _complete_ and _well-tested and benchmarked_\nsolution for common bit and byte-level routines, which can be used via the FFI\nor a wrapper, across many Haskell projects. This is what ``diablo`` aims to be.\n\n## What are the goals of this project?\n\n### Focus on Haskell integration\n\nOur goal is to make working with GHC Haskell as easy as possible: this is our\none, and _only_ goal. We do this by checking [Tier 1 and Tier\n2](https://gitlab.haskell.org/ghc/ghc/-/wikis/platforms) platforms via CI as far\nas is possible for correctness and buildability. If you can get GHC on your\nplatform and architecture, ``diablo`` should work too. We also provide easier\nintegration by having an 'amalgamation' single-file build, similar to SQLite. No\nneed to worry about system libraries, build systems or any of that: just drop\none file, note it in your dependencies, and go.\n\nNote that we use this both as a source of restriction _and_ a source of freedom.\nFor example, we don't support compilers other than GCC and Clang, as GHC cannot\nbe compiled without one or the other being available.\n\n### No dependencies\n\nNothing outside of the C11-specified stdlib and one of our supported compilers\nis required. The only dependencies ``diablo`` has are for tests and benchmarks:\nif you don't need those, you don't need the dependencies for them either. We\nalso painstakingly avoid any stdlib extras (like the ones provided by `glibc`\nfor example), and our CI checks against multiple stdlibs (including `musl`) to\nbe sure.\n\n### Documentation and performance\n\nC is a nightmare of correctness issues even at the best of times, for experts.\nHaskellers can and should have better things to do than figure out why the C\nstandard decided to be delusional that decade ago. Furthermore, even though C is\nknown for performance, what 'performance' means can vary on things like compiler\nchoice, phase of the moon and whether Cthulhu hates you today. This cannot and\nshould not be any business of any Haskell programmers - things should just go\nfast.\n\n``diablo`` aims to take away _all_ of these concerns. We provide detailed\ndocumentation, as well as tests and benchmarks, using a modern build system and\ntools, so that you can be sure that things are both correct _and_ fast, without\nhaving to lift a finger. This includes SIMD support: whenever possible, we\nprovide SIMD acceleration, so you don't have to.\n\n### Batteries included\n\nBit and byte-bashing operations are surprisingly common, showing up in areas\nlike text processing, serialization, cryptography, compression, and probably a\ndozen other things. Unfortunately, Haskell's provision of such primitives is\nlimited, particularly where byte _sequences_ are concerned. This forces\nHaskellers to either write their own, or try using someone else's, with their\nown peculiar caveats.\n\nWe aim to provide as many bit and byte-oriented operations as is reasonable\nhere, including many operations on sequences. If you feel we could add\nsomething, please suggest it!\n\n## How do I use this?\n\nThe easiest thing to do is to drop the `amalgamation/diablo.c` file into your\nHaskell project, then add it to your `c-sources`. You can then use the FFI\ndirectly, with the confidence that things will Just Work.\n\nIf you want to use this as a _C_ library, your best bet is a [Meson\nsubproject](https://mesonbuild.com/Subprojects.html).\n\n## How do I build this?\n\nWe use [Meson](https://mesonbuild.com/index.html). We support _only_ GCC and\nClang as compilers: in particular, should you want to build on Windows, the only\nsupported configuration is via MinGW.\n\nIf you also want to build the tests, you will need the following Python\ndependencies:\n\n* [CFFI](https://cffi.readthedocs.io/en/latest)\n* [Hypothesis](https://hypothesis.readthedocs.io/en/latest/index.html)\n\nIf you want to build the benchmarks, you will also need the following:\n\n* [PyTest](https://docs.pytest.org/en/6.2.x)\n* [pytest-benchmark](https://pytest-benchmark.readthedocs.io/en/stable/index.html)\n\nUse the standard approach to building a Meson project: this will build both a\nstatic and a shared library by default.\n\n## What's your platform support?\n\nOur goal is supporting all of the [Tier 1 platforms for\nGHC](https://gitlab.haskell.org/ghc/ghc/-/wikis/platforms#tier-1-platforms),\nwith as many of the [Tier\n2](https://gitlab.haskell.org/ghc/ghc/-/wikis/platforms#tier-2-platforms) ones\nas reasonable. Furthermore, we aim to be stdlib-neutral insofar as possible. \n\nThe table below summarizes our efforts so far. We consider a\nplatform-architecture combo 'Supported' when we have CI to test it. 'WIP'\nindicates that this is theoretically testable, but isn't currently done, or is\nbeing worked on.\n\n|Architecture|Platform|Tier|Supported?|Stdlibs|\n|------------|--------|----|----------|-------|\n|x86         |Windows (MinGW)|1|No way to test|N/A|\n|x86-64      |Windows (MinGW)|1|Yes|`glibc`|\n|x86         |Linux|1|No way to test|N/A|\n|x86-64      |Linux|1|Yes|`glibc`, `musl`|\n|x86-64      |MacOS|1|Yes|`libc`|\n|aarch64     |Linux|2|Yes|`glibc`, `musl`|\n|x86         |FreeBSD|2|WIP|N/A|\n|x86-64      |FreeBSD|2|WIP|N/A|\n|x86         |OpenBSD|2|No way to test|N/A|\n|x86         |Solaris|2|No way to test|N/A|\n|x86         |MacOS|2|No way to test|N/A|\n|x86-64      |OpenBSD|2|No way to test|N/A|\n|x86-64      |DragonFly|2|No way to test|N/A|\n|PowerPC     |Linux|2|No way to test|N/A|\n|PowerPC     |AIX|2|No way to test|N/A|\n|PowerPC64   |Linux|2|No way to test|N/A|\n|PowerPC64le |Linux|2|Yes|`glibc`, `musl`|\n|Sparc       |Linux|2|No way to test|N/A|\n|IA-64       |Linux|2|No way to test|N/A|\n|Alpha       |Linux|2|No way to test|N/A|\n|HPPA        |Linux|2|No way to test|N/A|\n|S/390       |Linux|2|Yes|`glibc`, `musl`|\n|m68k        |Linux|2|No way to test|N/A|\n|mips        |Linux|2|No way to test|N/A|\n|mipsel      |Linux|2|No way to test|N/A|\n|ARMv7       |Linux|2|Yes|`glibc`, `musl`|\n|ARMel       |Linux|2|No way to test|N/A|\n|ARM         |iOS|2|No way to test|N/A|\n\nWe provide SIMDed implementations of most functionality on the following\narchitectures:\n\n* Aarch64 (theoretically any NEON platform should work, but we can't benchmark\n  it)\n* x86 (all SSE)\n* x86-64 (all SSE, AVX, AVX2)\n\nFor x86-64, we use runtime detection for SIMD instruction sets to allow for\nportable binaries.\n\n## What can I do with this?\n\nThe project is licensed Apache 2.0 (SPDX code\n[`Apache-2.0`](https://spdx.org/licenses/Apache-2.0.html)). For more details,\nplease see the `LICENSE.md` file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaskell-text%2Fdiablo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhaskell-text%2Fdiablo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaskell-text%2Fdiablo/lists"}