{"id":15956114,"url":"https://github.com/jsign/bufpool","last_synced_at":"2025-04-04T07:42:05.254Z","repository":{"id":74725583,"uuid":"325816796","full_name":"jsign/bufpool","owner":"jsign","description":"[]bytes manager using a bump-the-pointer allocator.","archived":false,"fork":false,"pushed_at":"2021-01-02T00:50:49.000Z","size":11,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-29T23:45:17.261Z","etag":null,"topics":["allocator","golang","memory-pool","performance"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jsign.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-31T14:32:11.000Z","updated_at":"2024-06-18T08:52:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"bc9591e4-22f4-4a26-945e-c22687779c75","html_url":"https://github.com/jsign/bufpool","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jsign%2Fbufpool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jsign%2Fbufpool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jsign%2Fbufpool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jsign%2Fbufpool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jsign","download_url":"https://codeload.github.com/jsign/bufpool/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247142041,"owners_count":20890652,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["allocator","golang","memory-pool","performance"],"created_at":"2024-10-07T13:28:30.205Z","updated_at":"2025-04-04T07:42:05.227Z","avatar_url":"https://github.com/jsign.png","language":"Go","readme":"# bufpool\n\n`bufpool` manages a set of []bytes in a `sync.Pool` to provide smaller \n[]byte subslices.\n\n## Usage\nSample usage:\n```\npool := bufpool.New(1024) // Size of shards\nbuf := pool.Make(15)\n// buf.B is an unzeroed []byte with len/cap 15\nbuf.Clear() // if necessary; buf may contain data from previous uses\nbuf.Done() // return to pool for future use\n```\n\n## How this compares to usual `sync.Pool` solutions?\nUsing a single `sync.Pool` of []bytes has known problems, like receiving a low capacity slice, infecting the pool with big slices produced by even rare outliers, etc.\n\nA usual solution is to bucketize requests using different pools, which prevents outliers from infecting the pool with worst-case slices of high capacity and thus wasting memory.\n\nThis library takes a different approach. It manages a single `sync.Pool` of _big_[]bytes called _shards_. A request of []byte of size _n_  results in a slice from a shard in the pool. It is effectively a bump-the-pointer allocator within each shard.\n\n## Configuration\nThis library has two knobs:\n1. `sz`: The size of the shards.\n2. `maxRetries` (opt, default `1`): Maximum retries to find a suitable shard before making a new one.\n\n## How it works\nWhenever a new request of size `n` is received, a shard is pulled from the `sync.Pool` and checked if it contains enough free space for the request. If that is not the case, it will retry `maxRetries` times to see if other shards can fit the request. A new shard is created if none is found.\n\nSince each shard behaves as a bump-the-pointer allocator, ideally, subslices need to be returned fast to avoid bad consequences, like:\n1. A shard will be alive and not be GCed since a single subslice is still holding a reference to it.\n2. The ratio of used vs. free space of a shard will decrease rapidly since a single subslice prevents the shard pointer from starting from zero again.\n\nA shard pointer moved to zero-position when all subslices were returned, and a request cannot be fulfilled.\n\n## Tradeoffs\nThe two available knobs produce different tradeoffs to the allocator.\n\nIf the shard size is too small, it behaves as a direct `make` with extra overhead. If the shard size is too big, it will waste memory since a portion of the shard will not be ever subsliced.\n\nRegarding `maxRetries`, whenever a shard is pulled from the pool, it might not contain enough free space to generate the needed subslice size. At this point, the allocator has to create a new shard or try another pooled shard that might have enough free space.\n\nA low value will only try a few times (minimum of 1) to find a suitable shard and move to create a new shard if needed. This might create new shards when other shards in the pool also have enough free space for the needed size.\n\nA big `maxRetries` will inspect all existing shards for free space before creating a new shard, thus only allocating a new shard if necessary. On the other hand, it may increase latency/contention due to doing multiple `Get` from `sync.Pool`. It might also make a high concurrency scenario worse since pools are not returned to the pool until a request has finished, thus potentially creating new shards when unreturned ones might be fine to use.\n\nThe above analysis is purely theoretical and most probably useless to make decisions. The only way to know the correct values for each knob depends on the use-case and most probably will involve doing benchmarks to find optimal values. See the _Benchmark_ section for some naive simulation results.\n\n## Benchmarks\n\nThis section contains available benchmark results.\n\n### BenchmarkVsMalloc\nThis benchmark compares a single (thus, non-concurrent) client for different sizes (4, 64, 512, 16k bytes):\n- `malloc`: Using `make` directly.\n- `overhead`: Asking for a size greater than shard size results in a direct `make` call and thus measures the overhead cost compared to the above case.\n- `reuse`: Ask-and-return serially.  \n\n```\ngoos: linux\ngoarch: amd64\npkg: github.com/jsign/bufpool\nBenchmarkVsMalloc/4/malloc-16           100000000               16.0 ns/op             4 B/op          1 allocs/op\nBenchmarkVsMalloc/4/overhead-16         67191996                19.1 ns/op             4 B/op          1 allocs/op\nBenchmarkVsMalloc/4/reuse-16            19410166                53.8 ns/op             0 B/op          0 allocs/op\nBenchmarkVsMalloc/64/malloc-16          27107473                43.0 ns/op            64 B/op          1 allocs/op\nBenchmarkVsMalloc/64/overhead-16        24714236                49.3 ns/op            64 B/op          1 allocs/op\nBenchmarkVsMalloc/64/reuse-16           20522583                54.3 ns/op             0 B/op          0 allocs/op\nBenchmarkVsMalloc/512/malloc-16          8445880               138 ns/op             512 B/op          1 allocs/op\nBenchmarkVsMalloc/512/overhead-16        8599345               140 ns/op             512 B/op          1 allocs/op\nBenchmarkVsMalloc/512/reuse-16          18668682                54.3 ns/op             0 B/op          0 allocs/op\nBenchmarkVsMalloc/16384/malloc-16         355327              3334 ns/op           16384 B/op          1 allocs/op\nBenchmarkVsMalloc/16384/overhead-16       340364              3293 ns/op           16384 B/op          1 allocs/op\nBenchmarkVsMalloc/16384/reuse-16        20684917                54.8 ns/op             0 B/op          0 allocs/o\n```\n\n### BenchmarkChaos\nThis benchmark creates a scenario of concurrent calls with some non-uniform `Make` and `Done` calls, as _simulate_ some real use-case, with different shard retries to see the impact in allocations.\n\n```\ngoos: linux\ngoarch: amd64\npkg: github.com/jsign/bufpool\nBenchmarkChaos/0-16          134           8914546 ns/op        18926198 B/op     200415 allocs/op\nBenchmarkChaos/1-16          249           5057267 ns/op         2493774 B/op      21583 allocs/op\nBenchmarkChaos/2-16          234           5225900 ns/op         1411009 B/op       9802 allocs/op\nBenchmarkChaos/4-16          241           4752396 ns/op          850502 B/op       3594 allocs/op\nBenchmarkChaos/8-16          250           4656146 ns/op          608421 B/op        925 allocs/op\n```\nThe benchmark results should only be compared in a relative sense since the benchmark scenario adds work overhead unrelated to the allocator.\n\nThe benchmark is a synthetic scenario to show some numbers on available knobs.\n\n## Credits\nThis repository is based on a Playground snippet created by Josh Bleecher Snyder (@josharian). His original work can be found unchanged in the second commit of this repo.\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjsign%2Fbufpool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjsign%2Fbufpool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjsign%2Fbufpool/lists"}