{"id":13614154,"url":"https://github.com/couchbase/go-slab","last_synced_at":"2026-04-10T00:01:53.969Z","repository":{"id":8801401,"uuid":"10495741","full_name":"couchbase/go-slab","owner":"couchbase","description":"slab allocator in go","archived":false,"fork":false,"pushed_at":"2024-12-17T02:07:42.000Z","size":84,"stargazers_count":377,"open_issues_count":0,"forks_count":39,"subscribers_count":33,"default_branch":"master","last_synced_at":"2024-12-17T03:24:58.791Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/couchbase.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-06-05T05:36:29.000Z","updated_at":"2024-12-17T02:07:45.000Z","dependencies_parsed_at":"2022-09-02T10:51:52.325Z","dependency_job_id":null,"html_url":"https://github.com/couchbase/go-slab","commit_stats":null,"previous_names":["couchbaselabs/go-slab"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/couchbase%2Fgo-slab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/couchbase%2Fgo-slab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/couchbase%2Fgo-slab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/couchbase%2Fgo-slab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/couchbase","download_url":"https://codeload.github.com/couchbase/go-slab/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248760515,"owners_count":21157374,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T20:00:57.670Z","updated_at":"2025-10-07T15:21:12.945Z","avatar_url":"https://github.com/couchbase.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# go-slab - slab allocator in go\n\nA slab allocator library in the Go Programming Language.\n\n[![GoDoc](https://godoc.org/github.com/steveyen/go-slab?status.svg)](https://godoc.org/github.com/steveyen/go-slab) [![Build Status](https://drone.io/github.com/steveyen/go-slab/status.png)](https://drone.io/github.com/steveyen/go-slab/latest) [![Coverage Status](https://coveralls.io/repos/steveyen/go-slab/badge.png)](https://coveralls.io/r/steveyen/go-slab)\n\n# Who is this for\n\nThis library may be interesting to you if you wish to reduce garbage\ncollection (e.g. stop-the-world GC) performance issues in your golang\nprograms, allowing you to switch to explicit byte array memory\nmanagement techniques.\n\nThis can be useful, for example, for long-running server programs that\nmanage lots of in-memory data items, such as caches and databases.\n\n# Example usage\n\n    arena := NewArena(48,         // The smallest slab class \"chunk size\" is 48 bytes.\n                      1024*1024,  // Each slab will be 1MB in size.\n                      2,          // Power of 2 growth in \"chunk sizes\".\n                      nil)        // Use default make([]byte) for slab memory.\n\n    var buf []byte\n\n    buf = arena.Alloc(64)         // Allocate 64 bytes.\n      ... use the buf ...\n    arena.DecRef(buf)             // Release our ref-count when we're done with buf.\n\n    buf = arena.Alloc(1024)       // Allocate another 1K byte array.\n      ... use the buf ...\n    arena.AddRef(buf)             // The buf's ref-count now goes to 2.\n      ... use the buf some more ...\n    arena.DecRef(buf)\n      ... still can use the buf since we still have 1 ref-count ...\n    arena.DecRef(buf)             // We shouldn't use the buf after this last DecRef(),\n                                  // as the library might recycle it for a future Alloc().\n\n# Design concepts\n\nThe byte arrays ([]byte) that are allocated by this library are\nreference-counted.  When a byte array's reference count drops to 0, it\nwill be placed onto a free-list for later re-use.  This can reduce the\nneed to ask the go runtime to allocate new memory and perhaps delay\nthe need for a full stop-the-world GC.\n\nThe AddRef()/DecRef() functions use slice/capacity math instead of\n\"large\" additional tracking data structures (e.g., no extra\nhashtables) in order to reach the right ref-counter metadata.\n\nThis implementation also does not use any of go's \"unsafe\"\ncapabilities, allowing it to remain relatively simple.\n\nMemory is managed via a simple slab allocator algorithm.  See:\nhttp://en.wikipedia.org/wiki/Slab_allocation\n\nEach arena tracks one or more slabClass structs.  Each slabClass\nmanages a different \"chunk size\", where chunk sizes are computed using\na simple \"growth factor\" (e.g., the \"power of 2 growth\" in the above\nexample).  Each slabClass also tracks zero or more slabs, where every\nslab tracked by a slabClass will all have the same chunk size.  A slab\nmanages a (usually large) continguous array of memory bytes (1MB from\nthe above example), and the slab's memory is subdivided into many\nchunks of the same chunk size.  All the chunks in a new slab are\nplaced on a free-list that's part of the slabClass.\n\nWhen Alloc() is invoked, the first \"large enough\" slabClass is found,\nand a chunk from the free-list is taken to service the allocation.  If\nthere are no more free chunks available in a slabClass, then a new\nslab (e.g., 1MB) is allocated, chunk'ified, and the request is\nprocessed as before.\n\n# Concurrency\n\nThe Arena returned from NewArena() is not concurrency safe.\nPlease use your own locking.\n\n# Chainability\n\nThe []byte buf's can be chained via the SetNext()/GetNext() functions.\nThis may be useful for developers wishing to reduce fragmentation when\nthey have wildly varying byte array sizes.\n\nFor example, a server cache may need to manage many items whose sizes\nrange from small to large (16 bytes to 1MB).  Instead of invoking\nArena.Alloc() on the exact item size, the developer may wish to\nconsider slicing an item into many more smaller 4KB byte arrays.\n\nFor a 1MB item, for example, the application can instead invoke\nArena.Alloc(4096) for 256 times and use the Arena.SetNext() function\nto chain those smaller 4KB buffers together.  By slicing memory into\nuniform-sized, smaller-sized buffers, there may be less fragmentation\nand better overall re-use of slabs.  Additionally, the last []byte\nbuffer in the chain may be smaller than 4KB to not waste space.\n\n# Application specific slab memory allocator\n\nThe NewArena() function takes an optional malloc() callback function,\nwhich will be invoked whenever the arena needs more memory for a new\nslab.  If the malloc() func is nil, the arena will default to using\nthe builtin make([]byte, sizeNeeded).\n\nAn application-specific malloc() func can be useful for tracking\nand/or limiting the amount of slab memory that an Arena uses.  It can\nbe also used by advanced applications to supply mmap()'ed memory to an\nArena.\n\n# Rules\n\n* You need to invoke AddRef()/DecRef() with the exact same buf\n  that you received from Alloc(), from the same arena.\n* Don't call Alloc() with a size greater than the arena's slab size.\n  e.g., if your slab size is 1MB, then Alloc(1024 * 1024 + 1) will fail.\n* Careful with your ref-counting -- that's the fundamental tradeoff\n  with now trying to avoid GC.\n* Do not grow or append() on the slices returned by Alloc().\n* Do not use cap() on slices returned by Alloc(), as that has\n  information / abstraction \"leakage\" and should not be depended on.\n\n# LICENSE\n\nApache 2 license.\n\n# Testing\n\nUnit test code coverage, as of version 0.0.0-42-g60296ca, is 99.4%.\n\n# TODO\n\n* Currently, slabs that are allocated are never freed.\n* Memory for one slabClass is never reassigned to another slabClass.\n  Memory reassignment might be useful whenever data sizes of items in\n  long-running systems change over time.  For example, sessions in an\n  online game may initially fit fine into a 1K slab class, but start\n  getting larger than 1K as long time players acquire more inventory.\n  Meanwhile, most of the slab memory is \"stuck\" in the 1K slab class\n  when it's now needed in the 2K slab class.  The chainability features\n  of go-slab, of note, should also be considered in these cases.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcouchbase%2Fgo-slab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcouchbase%2Fgo-slab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcouchbase%2Fgo-slab/lists"}