{"id":16008119,"url":"https://github.com/ckerr/buffy","last_synced_at":"2025-10-08T00:03:47.292Z","repository":{"id":138268214,"uuid":"276139617","full_name":"ckerr/buffy","owner":"ckerr","description":"An embeddable, MIT-licensed, C-language, zero-dependency memory buffer class inspired by libevent's evbuffer. It consists of just a few files that can be dropped into your own project as-is.","archived":false,"fork":false,"pushed_at":"2020-10-22T07:58:59.000Z","size":121,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-08T00:02:59.432Z","etag":null,"topics":["buffers","c","cpp","endianness","evbuffer","libev","libevent","libuv","memory-management","networking","string-builder","strings"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ckerr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-06-30T15:32:10.000Z","updated_at":"2022-04-18T01:20:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"7540c988-1d21-4edb-b065-b31f4d9d8a70","html_url":"https://github.com/ckerr/buffy","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ckerr/buffy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckerr%2Fbuffy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckerr%2Fbuffy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckerr%2Fbuffy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckerr%2Fbuffy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ckerr","download_url":"https://codeload.github.com/ckerr/buffy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckerr%2Fbuffy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278866931,"owners_count":26059671,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["buffers","c","cpp","endianness","evbuffer","libev","libevent","libuv","memory-management","networking","string-builder","strings"],"created_at":"2024-10-08T12:23:57.311Z","updated_at":"2025-10-08T00:03:47.263Z","avatar_url":"https://github.com/ckerr.png","language":"C++","readme":"# libbuffy\n\nlibbuffy is an embeddable, MIT-licensed, C-language, zero-dependency\nmemory buffer class inspired by libevent's `evbuffer`. It consists of\njust a few files that can be dropped into your own project as-is.\n\nSome common uses:\n\n* A pipeline between data producers and consumers\n* Queueing data that's been received or about to be sent over the network\n* Building strings or data buffers without worrying about memory management\n\nBuffy is designed to be as efficient as possible despite being a\ngeneral-purpose tool. It avoids unnecessary memory allocations and\ncopies when possible.\n\n## Building libbuffy\n\n```sh\n$ git clone https://github.com/ckerr/buffy.git\n$ cd buffy\n$ git submodule update --init\n$ mkdir build\n$ cd build\n$ cmake ..\n$ make\n```\n\n## Concepts: Pages, Content, and Space\n\nbfy buffers are implemented using an array of separate pages, where a\n**page** is a chunk of contiguous memory. Page memory that's been written\ninto via `bfy_buffer_add*()` is the **content**, while the as-yet-unused\nmemory at the end of the last page is the buffer's free **space**. These\nterms are reflected in the API, e.g. `bfy_buffer_get_content_len()`.\n\nMost of the time, no special steps are needed for efficient use.\nbfy will use space available when possible and recycle leftover memory\nfrom content that's already been consumed by `bfy_buffer_remove*()` or\n`bfy_buffer_drain*()`.\n\nSometimes, though, you may have data with special constraints:\nit may be read-only, or it may be memory-managed by someone else\nand bfy shouldn't free it, or you may want to transfer it from a different\nbuffer. You can add these with the functions `bfy_buffer_add_reference()`,\n`bfy_buffer_add_readonly()`, or `bfy_buffer_add_buffer()`. These all create\nnew pages in the buffer's array of pages so pre-existing content\ncan be embedded into the buffer without the overhead of cloning it.\n\n## API\n\n### Life Cycle\n\nBuffers can be instantiated on the heap:\n\n```c\nbfy_buffer* bfy_buffer_new(void);\nvoid bfy_buffer_free(bfy_buffer*);\n```\n\nIf you're building bfy inside your own project such that bfy's ABI is\nnot a concern, you can also instantiate it on the stack or aggregate\nit in a struct to avoid another malloc/free pair for the `bfy_buffer`\nstruct itself:\n\n```c\nbfy_buffer bfy_buffer_init(void);\nvoid bfy_buffer_destruct(bfy_buffer*);\n```\n\n### Adding Data\n\n```c\nsize_t bfy_buffer_add(bfy_buffer* buf, void const* addme, size_t len);\nsize_t bfy_buffer_add_ch(bfy_buffer* buf, char addme);\nsize_t bfy_buffer_add_printf(bfy_buffer* buf, char const* fmt, ...);\nsize_t bfy_buffer_add_vprintf(bfy_buffer* buf, char const* fmt, va_list args);\nsize_t bfy_buffer_add_hton_u8 (bfy_buffer* buf, uint8_t  addme);\nsize_t bfy_buffer_add_hton_u16(bfy_buffer* buf, uint16_t addme);\nsize_t bfy_buffer_add_hton_u32(bfy_buffer* buf, uint32_t addme);\nsize_t bfy_buffer_add_hton_u64(bfy_buffer* buf, uint64_t addme);\n```\n\nOf these functions, `bfy_buffer_add()` is the key: it copies the specified\nbytes into the free space at the end of the last page, marking that space\nas content. If not enough space is available, more is allocated. The number\nof bytes added is returned.\n\nThe rest are convenience wrappers: `add_ch()` adds a single character;\n`add_printf() / add_vprintf()` are printf-like functions that add to the\nend of the buffer, and `add_hton_u*()` will convert numbers into big-endian\nnetwork byte order before ading them to he buffer.. Like `bfy_buffer_add()`,\nthese all try to append to the end of the current page.\n\n```c\nsize_t bfy_buffer_add_buffer(bfy_buffer* buf, bfy_buffer* src);\nsize_t bfy_buffer_add_readonly(bfy_buffer* buf, const void* data, size_t len);\nsize_t bfy_buffer_add_reference(bfy_buffer* buf, const void* data, size_t len,\n                                bfy_unref_cb* cb, void* user_data);\n```\n\nThese functions add pre-existing content into the buffer by embedding it\nrather than duplicating it.\n\n`add_buffer()` transfers the contents of one buffer to another.\n\n`add_reference()` adds a new page that embeds content managed outside of bfy.\nWhen bfy is done with the page, the unref callback passed to `add_reference()`\nis called.\n\n`add_readonly()` adds a new page that embeds read-only content.\n\n## Removing Data\n\n```c\nsize_t bfy_buffer_remove_range(bfy_buffer* buf, size_t begin, size_t end, void* setme);\nsize_t bfy_buffer_remove(bfy_buffer* buf, size_t len, void* setme);\n\nchar* bfy_buffer_remove_string(bfy_buffer* buf, size_t* len);\nint bfy_buffer_remove_ntoh_u8 (bfy_buffer* buf, uint8_t* setme);\nint bfy_buffer_remove_ntoh_u16(bfy_buffer* buf, uint16_t* setme);\nint bfy_buffer_remove_ntoh_u32(bfy_buffer* buf, uint32_t* setme);\nint bfy_buffer_remove_ntoh_u64(bfy_buffer* buf, uint64_t* setme);\n```\n\n`bfy_buffer_remove_range()` moves the [begin..end) subset of all\nthe buffer's content from the buffer into `setme` and returns the\nnumber of bytes moved. `bfy_buffer_remove()` is a convenience helper\nto remove the buffer's first `len` bytes. Just as content is added\nwith `bfy_buffer_add*()`, it is consumed with `bfy_buffer_remove*()`.\n\nThe others are also helpers: `bfy_buffer_remove_ntoh*()` read numbers\nfrom big-endian network byte order into the host machine's byte order,\nand `bfy_buffer_remove_string()` will remove the buffer into a\nnewly-allocated string.\n\n```c\nint bfy_buffer_remove_buffer(bfy_buffer* buf, bfy_buffer* tgt, size_t len);\n```\n\nThis moves the first `len` bytes of content from the source buffer to\nthe target buffer. Unnecessary memory copying is avoided as much as\npossible.\n\n```c\nsize_t bfy_buffer_drain_range(bfy_buffer* buf, size_t begin, size_t end);\nsize_t bfy_buffer_drain(bfy_buffer* buf, size_t len);\nsize_t bfy_buffer_drain_all(bfy_buffer* buf);\n```\n\n`bfy_buffer_drain_range()` removes the [begin..end) subset of the\nbuffer's content without copying it first. `bfy_buffer_drain_all()`\nand `bfy_buffer_drain()` are convenience helpers that drain the entire\nbuffer or its first `len` bytes, respectively.\n\n```c\nsize_t bfy_buffer_copyout_range(bfy_buffer const* buf, size_t begin, size_t end, void* setme);\nsize_t bfy_buffer_copyout(bfy_buffer const* buf, size_t len, void* setme);\n```\n\n`bfy_buffer_copyout_range()` copies the [begin..end) subset of the\nbuffer's contents into the provided `setme` buffer. `bfy_buffer_copyout()`\nis a convenience helper that copies the buffer's first `len` bytes.\n\n## Searching\n\n```c\nint bfy_buffer_search_range(bfy_buffer const* buf,\n                            size_t begin, size_t end,\n                            void const* needle, size_t needle_len,\n                            size_t* match);\n\nint bfy_buffer_search_all(bfy_buffer const* buf,\n                          void const* needle, size_t needle_len,\n                          size_t* match);\n\nint bfy_buffer_search(bfy_buffer const* buf, size_t len,\n                          void const* needle, size_t needle_len,\n                          size_t* match);\n\n```\n\n`bfy_buffer_search_range()` searches for a string inside the [begin..end)\nsubset of the buffer's content. `bfy_buffer_search_all()` and\n`bfy_buffer_search()` are convenience helpers that search the entire\nbuffer or its first `len` bytes, respectively.\n\n## Efficient Memory Management\n\n### Preallocating Space\n\nIf you know how much space you're going to use, it makes sense to tell bfy\nwith `bfy_buffer_ensure_space()` so that it can be preallocated once in a\nsingle page before you start filling it with `bfy_buffer_add*()`.\n\n`bfy_buffer_get_space_len()` returns how much space is available.\n\n```c\nsize_t bfy_buffer_get_space_len(bfy_buffer const* buf);\nint bfy_buffer_ensure_space(bfy_buffer* buf, size_t len);\n```\n\n### Peek / Reserve / Commit\n\nAs an alternative to `bfy_buffer_ensure_space()` + `bfy_buffer_add*()`,\nthe reserve/commit API lets you work directly a buffer's free space.\n`bfy_buffer_reserve_space(len)` preallocates `len` bytes of free space\nand returns its address. You can write directly to this memory or pass\nit along to a third party data generator, then commit your work to the\nbuffer with `bfy_buffer_commit_space()`. It's OK to commit fewer bytes\nthan you reserved, or even to skip committing altogether if you find\nthat you didn't need the space after all.\n\n`bfy_buffer_peek_space()` similar to `bfy_buffer_reserve_space()` but\nonly returns the free space that's already allocated. This is not often\nneeded but can be useful if you want to write as much content as possible\non the current page before adding a new page.\nAs with `bfy_buffer_reserve_space()`, you can commit some or all of your\nwork when done.\n\nIt is an error to do anything that changes the buffer while you're still\nworking on uncommitted memory. Changing the buffer can invalidate the\npointers returned by peek/reserve.\n\n```c\nstruct bfy_iovec bfy_buffer_reserve_space(bfy_buffer* buf, size_t len);\nstruct bfy_iovec bfy_buffer_peek_space(bfy_buffer* buf);\nsize_t bfy_buffer_commit_space(bfy_buffer* buf, size_t len);\n```\n\n### Contiguous / Non-contiguous Memory\n\nAs mentioned above in [Concepts](#concepts-pages-content-and-space),\nbuffers are made from a series of pages and a page is a contiguous\nblock of memory.\n\nIf you need to view multiple pages' worth of memory as a contiguous\nblock, `bfy_buffer_make_contiguous(len)` will ensure the first `len` bytes\nof content are in a single page. `bfy_buffer_make_all_contiguous()` is a\nconvenience helper to make the entire buffer a single page.\n\nInstead of the `contiguous()` functions, It is generally more efficient\nto walk through content with `bfy_buffer_peek()` or `bfy_buffer_peek_all()`\ninstead, since that can be done without moving data into a single page.\n\nUser code won't often need it, but `bfy_buffer_add_pagebreak()` can be\nused to force a pagebreak between calls to `bfy_buffer_add*()`.\n\n```c\nvoid* bfy_buffer_make_contiguous(bfy_buffer* buf, size_t len);\nvoid* bfy_buffer_make_all_contiguous(bfy_buffer* buf);\nsize_t bfy_buffer_peek(bfy_buffer const* buf, size_t size,\n                       struct bfy_iovec* vec_out, size_t vec_len);\nsize_t bfy_buffer_peek_all(bfy_buffer const* buf,\n                           struct bfy_iovec* vec_out, size_t vec_len);\nint bfy_buffer_add_pagebreak(bfy_buffer* buf);\n```\n\n## Comparison to `evbuffer`\n\nlibbuffy is inspired by\n[libevent](https://libevent.org/)'s [evbuffer](http://www.wangafu.net/~nickm/libevent-book/Ref7_evbuffer.html),\nso it's no surprise that they have similar APIs.\nBut the software world is full of event loop tools, so if you're using\n[glib](https://github.com/GNOME/glib) or\n[libuv](https://libuv.org/) or\n[libev](http://software.schmorp.de/pkg/libev.html) or\n[libhv](https://github.com/ithewei/libhv)\ninstead -- or none of the above! -- you may find libbuffy to be more accessible.\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fckerr%2Fbuffy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fckerr%2Fbuffy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fckerr%2Fbuffy/lists"}