{"id":20382434,"url":"https://github.com/mkchoi212/bitbuf","last_synced_at":"2025-09-10T08:08:11.928Z","repository":{"id":88876381,"uuid":"93473150","full_name":"mkchoi212/bitbuf","owner":"mkchoi212","description":"C API for creation and analysis of binary data","archived":false,"fork":false,"pushed_at":"2017-08-16T07:58:22.000Z","size":194,"stargazers_count":21,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-19T14:55:40.231Z","etag":null,"topics":["bit-algorithms","bit-array","bit-manipulation","bit-twiddling","c"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mkchoi212.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-06-06T03:43:46.000Z","updated_at":"2024-03-13T16:58:38.000Z","dependencies_parsed_at":"2023-06-12T22:45:20.102Z","dependency_job_id":null,"html_url":"https://github.com/mkchoi212/bitbuf","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/mkchoi212/bitbuf","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkchoi212%2Fbitbuf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkchoi212%2Fbitbuf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkchoi212%2Fbitbuf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkchoi212%2Fbitbuf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mkchoi212","download_url":"https://codeload.github.com/mkchoi212/bitbuf/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkchoi212%2Fbitbuf/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274429003,"owners_count":25283283,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-10T02:00:12.551Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bit-algorithms","bit-array","bit-manipulation","bit-twiddling","c"],"created_at":"2024-11-15T02:17:35.059Z","updated_at":"2025-09-10T08:08:11.920Z","avatar_url":"https://github.com/mkchoi212.png","language":"C","readme":"\u003cp align=\"center\"\u003e\n    \u003cimg src=\"./bitbuf_logo.gif\"\u003e\n\n  \u003ch3 align=\"center\"\u003ebitbuf\u003c/h3\u003e\n\n\n\u003cp align=\"center\"\u003e\n    C API for creation and analysis of binary data\n    \u003cbr\u003e\n    \u003cbr\u003e\n    \u003cimg src=\"https://travis-ci.org/mkchoi212/bitbuf.svg?branch=master\"\u003e\n    \u003ca href=\"https://codecov.io/gh/mkchoi212/bitbuf\"\u003e\u003cimg src=\"https://codecov.io/gh/mkchoi212/bitbuf/branch/master/graph/badge.svg\"\u003e\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/p\u003e\n\u003cbr\u003e\n\nbitbuf is a fast C API designed to help make the creation and analysis of binary data as simple and natural as possible.\n\nbitbuf structures can be constructed from integers, strings, or files.\nThey can also be sliced, appended, reversed, inserted and so on with simple functions.\nThey can also be searched, replaced, read from, and navigated with ease.\n\nThink of it as C++'s `bitset` on crack, with the size of your entire memory as the only limitation of its size.\n\n# How bitbuf works\n\nbitbuf is designed to be as lightweight as possible and can be considered to be just a list of binary digits. They are however stored efficiently - although there are a variety of ways of creating and viewing the binary data, bitbuf stores the byte data, and all views are calculated as needed.\n\nAll operations work under a struct named `bitbuf`. The struct is composed of three variables; `buf`, `alloc`, and `len`.\n\n`buf` is a pointer to an `unsigned char` byte array, which holds all the data. \n`alloc` is a variable that indicates how many bits have been allocated for usage behind the scenes.\n`len` then shows how much of the allocated space is actually being used to store the data.\n\n# Getting Started\nTyping `make` will generate a static library `libbitbuf.a` in your current directory. You may then move it to a directory of your choice that has been specified by a `LD_LIBRARY_PATH`.\n\n## Initialization\nFirst things first. Here is how you create a bitbuf.\n\n```c\nbitbuf b = BITBUF_INIT;\n```\n\nHere, `BITBUF_INIT` is a macro that initializes all variables within the `bitbuf` structure.\n**This is required at all times as C does not initialize variables with their default values.**\n\nOK, we've just created a bitbuf structure on the stack. Cool, but how do I initialize it with binary data stored in it?\n\nYou have several choices to choose from.\n\n- `init( size_t )` \n    - Initalize an empty buffer\n- `init_zero( size_t )` \n    - Initialize with `n` zeros.\n- `init_file( const char * )`\n    - Initialize with contents from a file\n- `init_str( const char * )`\n    - Initialize from strings\n- `init_sub( bitbuf *src )`\n    - Initialize with contents from another buffer\n\n\u003e Note that these functions allocate additional memory on the \"heap\" for storage and must be `free()`ed later to prevent memory leakage by calling\n\n```c\nbitbuf_release( \u0026buf );\n```\n\nOut of the five `init` methods, the most versatile method to create a bitbuf is with strings. So, I will use that as the main example.\n\n```c\nbitbuf b = BITBUF_INIT;\nbitbuf_init_str( \u0026b, \"0xdeadbeef 0b110\" );\n```\nHere, we just created a bitbuf of length 35 from a string that consists of both hexadecimal and binary strings. Note that the second segment of the input string is just 3 bits long. bitbuf handles all data that does not fall into byte units internally.\n\nThe supported prefixes for this function are `0x` for hexadecimal strings and `0b` for binary strings. \n\nThere are lots of things we can do with our new bitbuf, the simplest of which is to print them.\n\n## Representation\n```\n\u003e\u003e bitbuf_dump( \u0026b );\n0xdeadbeef 0b110\n```\n\nNote that I just passed the address of the bitbuf to the function as it takes a pointer to a bitbuf as its only argument.\n\nNow, when they are simply dumped to stdout, they are represented in the simplest hex | binary representation of themselves. If you prefer you can pick the representation that you want.\n\n```c\n\u003e\u003e char binStr[ b.len + 1 ];   // +1 char for the null terminating character\n\u003e\u003e bitbuf_bin( \u0026b, binStr );\n\u003e\u003e printf( \"%s\", binStr ); \n11011110101011011011111011101111110\n```\n\nHere, note that I created a char array that is later used to store the binary representation of the bitbuf. When converting bitbuf to their string representations, make sure to allocate a long enough array as not doing so will likely crash your program; here, we refer to bitbuf's `len` variable, which stores the number of bits contained in the buffer.\n\nTo get different representations of a bitbuf, you can use the functions below.\n\n- `bitbuf_bin`\n- `bitbuf_hex`\n- `bitbuf_rep` (mix of hex and bin for buffers of ambiguous length)\n- `bitbuf_ascii`\n- `bitbuf_num`\n\nOk, while we are at it, let's try one.\n\n```c\n\u003e\u003e char hexStr[ b.len / 4 + 1 ];\n\u003e\u003e bitbuf_hex( \u0026b, hexStr );\nhex: Cannot convert to hex unambiguously - not multiple of nibbles\nError 1\n```\n\nbitbuf `b` we just passed into the function is 35 bits long.\nThis means that since each hex digit is 4 bits long, there is no unambiguous way to represent it as a hex.\nWhen similar exceptions are met within the library, the program will output an error message to the stderr and exit.\n\n## Modifying bitbuf\nA bitbuf can be treated just like an array of bits. You can slice it, delete sections, append bitbufs and more.\n\nIf you ask for a single bit, an unsigned char is returned as it is the smallest type available in C and reminds the user once again that a bit can either be 0 or 1.\n\nTo join bitbufs, you can use `bitbuf_addbuf` or `bitbuf_addstr_[hex | bin ]`.\n\n```c\nbitbuf_addstr_hex( \u0026b, \"cafe\" )\n```\n\n## Finding and Replacing\n`bitbuf_find` is provided to search for binary patterns within a bitbuf. You can choose whether to search from the beginning or any bit position.\nIn addition, you can also specifiy the number of garbles (errors) allowed during the search.\n\nHere is a simple program that searches for a pattern within a bitbuf and prints the bit position of all matching occurences.\n\n```c\nbitbuf b = BITBUF_INIT;\nbitbuf_init_file( \u0026b, \"FILE_NAME\" );\n\nbitbuf pat = BITBUF_INIT;\nbitbuf_init_str( \u0026pat, \"0xcafe\" );\n\nint cur;\ncur = 0;\n\n// find returns -1 when no patterns are found\nwhile( ( cur = bitbuf_find( \u0026b, \u0026pat, 32, cur ) ) != -1 ) {\tprintf( \"%i\\n\", cur );\n    ++cur;\n}\n    \nbitbuf_release( \u0026b );\nbitbuf_release( \u0026pat );\n```\n\n# Example - Sieve of Eratosthenes\nThe sieve of Eratosthenes is an ancient (and very inefficient) method of finding prime numbers. The algorithm starts with the number 2 (which is a prime) and marks all of its multiples as not prime. It then continues with the next unmarked integer (which will also be prime) and marks all of its multiples as not prime.\n\nSo, to print all primes under a million you could write:\n\n```c\n// We will do a search until million\nconst size_t MAX_SCH = 1000000; \n              \n// Create million zero bits\nbitbuf buf = BITBUF_INIT;\nbitbuf_init_zero( \u0026buf, MAX_SCH );    \n        \nsize_t i, j;\nfor( i = 2; i \u003c MAX_LIM; ++i ) {\n    if( !bitbuf_getbit( \u0026buf, i ) ) {\n        printf( \"%i\\n\", i );\n        \n        // Set all multiples of current prime to not prime\n        for( j = i * 2; j \u003c MAX_SCH; j+=i ) \n            bitbuf_setbit( \u0026buf, j, 1 );    \n    }\n}\n\nbitbuf_release( \u0026buf );\n```\n\nOne reason you might want to use bitbuf for this purpose - instead of a plain array - is that the million bits only take up million bits in memory, whereas an array of integers would take up much more space. Try making an array with billion integers - it will fail unless you’ve got some really nice hardware. A billion element bitbuf is only **125MB** big.\n\n# TODO\n- Implement `bitbuf_init_num`\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkchoi212%2Fbitbuf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmkchoi212%2Fbitbuf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkchoi212%2Fbitbuf/lists"}