{"id":21126816,"url":"https://github.com/unit-io/bpool","last_synced_at":"2025-07-26T01:11:55.210Z","repository":{"id":57519015,"uuid":"247820654","full_name":"unit-io/bpool","owner":"unit-io","description":"Buffer pool with capacity in order to prevent from excess memory usage and CPU trashing.","archived":false,"fork":false,"pushed_at":"2020-09-06T00:57:30.000Z","size":51,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-21T05:42:21.016Z","etag":null,"topics":["bufferpool","memory-allocation"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/unit-io.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-16T21:33:54.000Z","updated_at":"2023-02-02T22:28:46.000Z","dependencies_parsed_at":"2022-09-26T18:00:51.825Z","dependency_job_id":null,"html_url":"https://github.com/unit-io/bpool","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unit-io%2Fbpool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unit-io%2Fbpool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unit-io%2Fbpool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unit-io%2Fbpool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/unit-io","download_url":"https://codeload.github.com/unit-io/bpool/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243573169,"owners_count":20312879,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bufferpool","memory-allocation"],"created_at":"2024-11-20T04:45:34.554Z","updated_at":"2025-03-14T11:43:47.668Z","avatar_url":"https://github.com/unit-io.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [![GoDoc](https://godoc.org/github.com/unit-io/bpool?status.svg)](https://pkg.go.dev/github.com/unit-io/bpool) [![Go Report Card](https://goreportcard.com/badge/github.com/unit-io/bpool)](https://goreportcard.com/report/github.com/unit-io/bpool)\n\n## Buffer pool prevent from excess memory usage and CPU trashing.\n\n## Quick Start\nTo import bpool from source code use go get command.\n\n\u003e go get -u github.com/unit-io/bpool\n\n## Usage\nUse buffer pool for writing incoming requests to buffer such as Put or Batch operations or use buffer pool while writing data to log file (during commit operation). The objective of creating BufferPool library with capacity is to perform initial writes to buffer without backoff until buffer pool reaches its target size. Buffer pool does not discard any Get or Write requests but it add gradual delay to it to limit the memory usage that can used for other operations such writing to log or db sync operations.\n\nDetailed API documentation is available using the [godoc.org](https://godoc.org/github.com/unit-io/bpool) service.\n\nMake use of the client by importing it in your Go client source code. For example,\n\nimport \"github.com/unit-io/bpool\"\n\nFollowing code snippet if executed without buffer capacity will consume all system memory and will cause a panic.\n\n```\n\tbuf := bytes.NewBuffer(make([]byte, 0, 2))\n\n\tdefer func() {\n\t\tif r := recover(); r != nil {\n\t\t\tfmt.Println(\"panics from blast\")\n\t\t}\n\t}()\n\n\tfor {\n\t\t_, err := buf.Write([]byte(\"create blast\"))\n\t\tif err != nil {\n\t\t\tfmt.Println(err.Error())\n\t\t\treturn\n\t\t}\n\t}\n\n```\n\nCode snippet to use BufferPool with capacity will limit usage of system memory by adding gradual delay to the requests and will not cause a panic.\n\n```\n  \tpool := bpool.NewBufferPool(1\u003c\u003c20, \u0026bpool.Options{MaxElapsedTime: 1 * time.Minute, WriteBackOff: true}) // creates BufferPool of 16MB target size\n\tbuf := pool.Get()\n\tdefer\tpool.Put(buf)\n\t\n\tfor {\n\t\t_, err := buf.Write([]byte(\"create blast\"))\n\t\tif err != nil {\n\t\t\tfmt.Println(err.Error())\n\t\t\treturn\n\t\t}\n\t}\n\n```\n\n\n### New Buffer Pool\nUse bpool.NewBufferPool() method and pass BufferSize parameter to create new buffer pool.\n\n```\n\tconst (\n\t\tBufferSize = 1\u003c\u003c30 // (1GB size)\n\t)\n\n\tpool := bpool.NewBufferPool(BufferSize, nil)\n\n```\n\n### Get Buffer\nTo get buffer from buffer pool use BufferPool.Get(). When buffer pool reaches its capacity Get method runs with gradual delay to limit system memory usage.\n\n```\n\t....\n\tvar buffer *bpool.Buffer\n\tbuffer = pool.Get()\n\n```\n\n### Writing to Buffer\nTo write to buffer use Buffer.Write() method.\n\n```\n\tvar scratch [8]byte\n\tbinary.LittleEndian.PutUint64(scratch[0:8], uint64(buffer.Size()))\n\n\tb.buffer.Write(scratch[:])\n\t....\n\n```\n\n### Reading from Buffer\nTo read buffer use Buffer.Bytes() method. This operation returns underline data slice stored into buffer.\n\n```\n\tdata := buffer.Bytes()\n\t...\n\n```\n\n### Put Buffer to Pool\nTo put buffer to the pool when finished using buffer use BufferPool.Put() method, this operation resets the underline slice. It also resets the buffer pool interval that was used to delay the Get operation if capacity is below the target size.\n\n```\n\tpool.Put(buffer)\n\t...\n\n```\n\nTo reset the underline slice stored to the buffer and continue using the buffer use Buffer.Reset() method instead of using BufferPool.Put() operation.\n\n```\n\tbuffer.Reset()\n\t....\n\n```\n\n## Contributing\nIf you'd like to contribute, please fork the repository and use a feature branch. Pull requests are welcome.\n\n## Licensing\nThis project is licensed under [MIT License](https://github.com/unit-io/bpool/blob/master/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funit-io%2Fbpool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funit-io%2Fbpool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funit-io%2Fbpool/lists"}