{"id":13667898,"url":"https://github.com/phip1611/simple-chunk-allocator","last_synced_at":"2025-12-30T05:41:03.740Z","repository":{"id":49942924,"uuid":"462882425","full_name":"phip1611/simple-chunk-allocator","owner":"phip1611","description":"A simple allocator written in Rust that manages memory in fixed-size chunks.","archived":true,"fork":false,"pushed_at":"2024-09-29T07:42:31.000Z","size":222,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-02T05:06:40.815Z","etag":null,"topics":["allocator","memory-management","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/phip1611.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-23T19:44:06.000Z","updated_at":"2024-09-29T07:43:48.000Z","dependencies_parsed_at":"2024-11-11T03:40:37.672Z","dependency_job_id":null,"html_url":"https://github.com/phip1611/simple-chunk-allocator","commit_stats":{"total_commits":58,"total_committers":2,"mean_commits":29.0,"dds":"0.12068965517241381","last_synced_commit":"f27bb1c92e2e42b6bb9d537baab1206f02bfc0ff"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phip1611%2Fsimple-chunk-allocator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phip1611%2Fsimple-chunk-allocator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phip1611%2Fsimple-chunk-allocator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phip1611%2Fsimple-chunk-allocator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/phip1611","download_url":"https://codeload.github.com/phip1611/simple-chunk-allocator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251035127,"owners_count":21526308,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["allocator","memory-management","rust"],"created_at":"2024-08-02T07:00:53.980Z","updated_at":"2025-12-12T15:04:38.819Z","avatar_url":"https://github.com/phip1611.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"# Simple Chunk Allocator\n\n**Disclaimer 2024-09-29:**\n\nI discourage the use of this library. Use it only as learning resource or so,\nbut please refrain from using it. It contains a few cases that produce UB.\n\nThere are better alternatives, such as https://crates.io/crates/talc.\n\n---\n\nA simple `no_std` allocator written in Rust that manages memory in fixed-size chunks/blocks. Useful for basic `no_std`\nbinaries where you want to manage a heap of a few megabytes without complex features such as paging/page table\nmanagement. Instead, this allocator gets a fixed/static memory region and allocates memory from there. This memory\nregion can be contained inside the executable file that uses this allocator. See examples down below.\n\n⚠ _Other allocators with different properties (for example better memory utilization but less\nperformance) do exist. The README of the repository contains a section that discusses how this allocator\nrelates to other existing allocators on \u003ccrates.io\u003e._ ⚠\n\n## TL;DR\n- ✅ `no_std` allocator with test coverage\n- ✅ uses static memory as backing storage (no paging/page table manipulations)\n- ✅ allocation strategy is a combination of next-fit and best-fit\n- ✅ reasonable fast with low code complexity\n- ✅ const compatibility (no runtime `init()` required)\n- ✅ efficient in scenarios where heap is a few dozens megabytes in size\n- ✅ user-friendly API\n\nThe inner and low-level `ChunkAllocator` can be used as `#[global_allocator]` with the synchronized wrapper type\n`GlobalChunkAllocator`. Both can be used with the `allocator_api` feature. The latter enables the usage in several\ntypes of the Rust standard library, such as `Vec::new_in` or `BTreeMap::new_in`. This is primarily interesting for\ntesting but may also enable other interesting use-cases.\n\nThe focus is on `const` compatibility. The allocator and the backing memory can get initialized during compile time\nand need no runtime `init()` call or similar. This means that if the compiler accepts it then the allocation will\nalso work during runtime. However, you can also create allocator objects during runtime.\n\nThe inner and low-level `ChunkAllocator` is a chunk allocator or also called fixed-size block allocator. It uses a\nmixture of the strategies next-fit and a best-fit. It tries to use the smallest gap for an allocation request to\nprevent fragmentation but this is no guarantee. Each allocation is a trade-off between a low allocation time and\npreventing fragmentation. The default chunk size is `256 bytes` but this can be changed as compile time const generic.\nHaving a fixed-size block allocator enables an easy bookkeeping algorithm through a bitmap but has as consequence that\nsmall allocations, such as `64 byte` will take at least one chunk/block of the chosen block size.\n\nThis project originates from my [Diplom thesis project](https://github.com/phip1611/diplomarbeit-impl). Since I\noriginally had lots of struggles to create this (my first ever allocator), I outsourced it for better testability and\nto share my knowledge and findings with others in the hope that someone can learn from it in any way.\n\n## Minimal Code Example\n```rust\n#![feature(const_mut_refs)]\n#![feature(allocator_api)]\n\nuse simple_chunk_allocator::{heap, heap_bitmap, GlobalChunkAllocator, PageAligned};\n\n// The macros help to get a correctly sized arrays types.\n// I page-align them for better caching and to improve the availability of\n// page-aligned addresses.\n\n/// Backing storage for heap (1Mib). (read+write) static memory in final executable.\n///\n/// heap!: first argument is chunk amount, second argument is size of each chunk.\n///        If no arguments are provided it falls back to defaults.\n///        Example: `heap!(chunks=16, chunksize=256)`.\nstatic mut HEAP: PageAligned\u003c[u8; 1048576]\u003e = heap!();\n/// Backing storage for heap bookkeeping bitmap. (read+write) static memory in final executable.\n///\n/// heap_bitmap!: first argument is amount of chunks.\n///               If no argument is provided it falls back to a default.\n///               Example: `heap_bitmap!(chunks=16)`.\nstatic mut HEAP_BITMAP: PageAligned\u003c[u8; 512]\u003e = heap_bitmap!();\n\n// please make sure that the backing memory is at least CHUNK_SIZE aligned; better page-aligned\n#[global_allocator]\nstatic ALLOCATOR: GlobalChunkAllocator =\n    unsafe { GlobalChunkAllocator::new(HEAP.deref_mut_const(), HEAP_BITMAP.deref_mut_const()) };\n\nfn main() {\n    // at this point, the allocator already got used a bit by the Rust runtime that executes\n    // before main() gets called. This is not the case if a `no_std` binary gets produced.\n    let old_usage = ALLOCATOR.usage();\n    let mut vec = Vec::new();\n    vec.push(1);\n    vec.push(2);\n    vec.push(3);\n    assert!(ALLOCATOR.usage() \u003e old_usage);\n\n    // use \"allocator_api\"-feature. You can use this if \"ALLOCATOR\" is not registered as\n    // the global allocator. Otherwise, it is already the default.\n    let _boxed = Box::new_in([1, 2, 3], ALLOCATOR.allocator_api_glue());\n}\n```\n\n## Another Code Example (Free Standing Linux Binary)\nThis is an excerpt. The code can be found in the GitHub repository in `freestanding-linux-example`.\n```rust\nstatic mut HEAP: PageAligned\u003c[u8; 256]\u003e = heap!(chunks = 16, chunksize = 16);\nstatic mut HEAP_BITMAP: PageAligned\u003c[u8; 2]\u003e = heap_bitmap!(chunks = 16);\n\n// please make sure that the backing memory is at least CHUNK_SIZE aligned; better page-aligned\n#[global_allocator]\nstatic ALLOCATOR: GlobalChunkAllocator\u003c16\u003e =\n    unsafe { GlobalChunkAllocator::\u003c16\u003e::new(HEAP.deref_mut_const(), HEAP_BITMAP.deref_mut_const()) };\n\n/// Referenced as entry by linker argument. Entry into the code.\n#[no_mangle]\nfn start() -\u003e ! {\n    write!(StdoutWriter, \"Hello :)\\n\").unwrap();\n    let mut vec = Vec::new();\n    (0..10).for_each(|x| vec.push(x));\n    write!(StdoutWriter, \"vec: {:#?}\\n\", vec).unwrap();\n    exit();\n}\n```\n\n## MSRV\nThis crate only builds with the nightly version of Rust because it uses many nightly-only features. I developed it\nwith version `1.61.0-nightly` (2022-03-05). Older nightly versions might work. So far there is no stable Rust\ncompiler version that compiles this.\n\n## Performance\nThe default CHUNK_SIZE is 256 bytes. It is a tradeoff between performance and efficient memory usage.\n\nI executed my example `bench` in release mode on an Intel i7-1165G7 CPU and a heap of `160MB` to get the results listed\nbelow. I used `RUSTFLAGS=\"-C target-cpu=native\" cargo run --release --example bench` to excute the benchmark with\nmaximum performance. The benchmark simulates a heavy usage of the heap in a single-threaded program with many random\nallocations and deallocations. The benchmark stops when the heap is close to 100%. The allocations vary in their alignment.\nThe table below shows the results of this benchmark as number of clock cycles.\n\n*Info: Since I measured those values, I slightly changed the benchmark.*\n\n| Chunk Size    | # Chunks | # allocations | # deallocations | median | average  | min | max   |\n|---------------|----------|---------------|-----------------|--------|----------|-----|-------|\n| 128           | 1310720  | 68148         | 47915           | 955    | 1001     | 126 | 57989 |\n| 256 [DEFAULT] | 655360   | 71842         | 51744           | 592    | 619      | 121 | 53578 |\n| 512           | 327680   | 66672         | 46858           | 373    | 401      | 111 | 54403 |\n\nThe results vary slightly because each run gets influenced by some randomness. One can see that the performance\ngets slower with a growing number of chunks. Increasing the chunk size reduces the size of the bookkeeping bitmap which\naccelerates the lookup. However, a smaller chunk size occupies less heap when only very small allocations are required.\n\nNote that performance is better than listed above when the heap is used less frequently and does not run full.\n\n## Differences to Other Allocators\n### good_memory_allocator (galloc)\n**Update November 2022**: I recently found [this new project](https://github.com/MaderNoob/galloc)\nand, from a first glance, I recommend to use this crate instead of mine for production usage. It has\nimpressive performance and heap utilization at the costs of more complicated code. The repository\nincludes interesting performance numbers from galloc, simple-chunk-allocator (this crate), and\nlinked-list-allocator.\n\n### linked-list-allocator\n**Update November 2022**: I wrote this paragraph before I found out about galloc. I left it\nunchanged.\n\nThe [linked-list-allocator](https://github.com/rust-osdev/linked-list-allocator) is among the few\nother well-suited and maintained general-purpose no-std allocator I could find on crates.io.\n\n**Advantages of my chunk allocator:**\n- much faster median allocation time\n- much faster average allocation time [ONLY IF HEAP IS NOT CLOSE TO BEEING FULL]\n- optimized realloc in certain cases (almost a no-op in some situations)\n- uses relatively easy algorithm (but needs dedicated heap and book-keeping backing storage)\n\n**Advantages of *linked-list-allocator*:**\n- better memory utilization (less fragmentation)\n- better worst-case allocation time in most test runs\n- better average allocation time [ONLY IF HEAP IS CLOSE TO BEEING FULL]\n- only needs a single chunk of memory and manages the heap with the backing-memory itself\n\n**Benchmark Comparision**:\nI ran `$ cargo run --example bench --release` against both allocators and obtained the following results. The benchmark\nperforms random allocations of different sizes and alignments and also deallocates some of the older allocations. Over\ntime, the heap becomes full, which is why the number of successful allocations has a higher delta to the attempted\nallocations.\n\nRuntime: 1s (most time lot's of heap available)\n```\nRESULTS OF BENCHMARK: Chunk Allocator\n     53360 allocations,  16211 successful_allocations,  37149 deallocations\n    median=   878 ticks, average=  1037 ticks, min=   158 ticks, max= 7178941 ticks\n\nRESULTS OF BENCHMARK: Linked List Allocator\n     31627 allocations,   9374 successful_allocations,  22253 deallocations\n    median= 18582 ticks, average= 44524 ticks, min=    71 ticks, max=44126026 ticks\n```\n\nWe see that as long as most allocations are done on a heap with lots of space available, the chunk\nallocator is faster in median and average performance.\n\nRuntime: 10s (most time heap almost full)\n```\nRESULTS OF BENCHMARK: Chunk Allocator\n     74909 allocations,  23753 successful_allocations,  51156 deallocations\n    median=   961 ticks, average=273362 ticks, min=   167 ticks, max=53330953 ticks\n\nRESULTS OF BENCHMARK: Linked List Allocator\n     81884 allocations,  24792 successful_allocations,  57092 deallocations\n    median=100196 ticks, average=179495 ticks, min=    69 ticks, max=43937820 ticks\n```\n\nWe see that when the heap is almost full, the chunk allocator has a faster median performance\nbut a worse worst-case allocation time. The linked list allocator performs better on average (but not on median)\nwhen it is close to beeing full.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphip1611%2Fsimple-chunk-allocator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fphip1611%2Fsimple-chunk-allocator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphip1611%2Fsimple-chunk-allocator/lists"}