{"id":20181517,"url":"https://github.com/antoniosarosi/rulloc","last_synced_at":"2025-08-21T20:32:16.107Z","repository":{"id":64271376,"uuid":"569515245","full_name":"antoniosarosi/rulloc","owner":"antoniosarosi","description":"General purpose memory allocator written in Rust.","archived":false,"fork":false,"pushed_at":"2023-01-03T20:25:51.000Z","size":191,"stargazers_count":106,"open_issues_count":1,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-13T20:17:32.204Z","etag":null,"topics":["linked-list","low-level","memory-allocation","mmap","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antoniosarosi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-11-23T02:04:04.000Z","updated_at":"2025-07-02T17:03:59.000Z","dependencies_parsed_at":"2023-01-15T07:45:33.788Z","dependency_job_id":null,"html_url":"https://github.com/antoniosarosi/rulloc","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/antoniosarosi/rulloc","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoniosarosi%2Frulloc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoniosarosi%2Frulloc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoniosarosi%2Frulloc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoniosarosi%2Frulloc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antoniosarosi","download_url":"https://codeload.github.com/antoniosarosi/rulloc/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoniosarosi%2Frulloc/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271026670,"owners_count":24687073,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["linked-list","low-level","memory-allocation","mmap","rust"],"created_at":"2024-11-14T02:35:50.699Z","updated_at":"2025-08-21T20:32:15.796Z","avatar_url":"https://github.com/antoniosarosi.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Rulloc\n\nRulloc (Rust Allocator) is a general purpose memory allocator written in Rust.\nIt implements\n[`std::alloc::Allocator`](https://doc.rust-lang.org/std/alloc/trait.Allocator.html)\nand\n[`std::alloc::GlobalAlloc`](https://doc.rust-lang.org/stable/std/alloc/trait.GlobalAlloc.html)\ntraits. All memory is requested from the kernel using\n[`mmap`](https://man7.org/linux/man-pages/man2/mmap.2.html) syscalls on Linux\nand\n[`VirtualAlloc`](https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualalloc)\non Windows. You can run the examples as follows:\n\n```bash\ncargo run --example standalone\ncargo run --example global\ncargo run --example buckets\ncargo run --example aligned\n```\n\nRun the tests:\n\n```bash\ncargo test\n```\n\nRun with [Miri](https://github.com/rust-lang/miri):\n\n```bash\ncargo miri test\ncargo miri run --example standalone\ncargo miri run --example buckets\ncargo miri run --example aligned\n```\n\nGlobal allocator example doesn't work with Miri, see\n[`examples/global.rs`](./examples/global.rs).\n\n## Implementation\n\nI started this project for learning purposes, and I know the best way to make\nsure you understand something is explaining it to others. So there's plenty of\ndocumentation and ASCII diagrams throughout the codebase if you're interested\nin how memory allocators work internally. Start by reading\n[`src/allocator.rs`](./src/allocator.rs) for a quick overview and you'll be\nredirected to the rest of files through\n[intra-doc links](https://doc.rust-lang.org/rustdoc/write-documentation/linking-to-items-by-name.html).\n\n### [Block](./src/block.rs)\n\nIf you don't want to scroll through hundreds of lines of code, this is how a\nmemory block looks like:\n\n\n```text\n+----------------------------+\n| pointer to next block      |   \u003c------+\n+----------------------------+          |\n| pointer to prev block      |          |\n+----------------------------+          |\n| pointer to block region    |          |\n+----------------------------+          | Block Header\n| block size                 |          |\n+----------------------------+          |\n| is free flag (1 byte)      |          |\n+----------------------------+          |\n| padding (struct alignment) |   \u003c------+\n+----------------------------+\n|       Block content        |   \u003c------+\n|            ...             |          |\n|            ...             |          | Addressable content\n|            ...             |          |\n|            ...             |   \u003c------+\n+----------------------------+\n```\n\n### [Region](./src/region.rs)\n\nAll blocks belong to a memory region, which is a contiguous chunk of memory\nthat can store multiple pages. In other words, the size of each region is a\nmultiple of the virtual memory page size on the current platform, and each\nregion contains a linked list of blocks:\n\n```text\n+--------+--------------------------------------------------+\n|        | +-------+    +-------+    +-------+    +-------+ |\n| Region | | Block | -\u003e | Block | -\u003e | Block | -\u003e | Block | |\n|        | +-------+    +-------+    +-------+    +-------+ |\n+--------+--------------------------------------------------+\n```\n\n### [Bucket](./src/bucket.rs)\n\nThe allocator can be configured at compile time with multiple allocation buckets\nof different sizes in order to reduce fragmentation. Each bucket stores a linked\nlist of memory regions and a free list, which is basically a linked list of only\nfree blocks:\n\n```text\n                   Next free block in the free list            Next free block\n               +--------------------------------------+   +-----------------------+\n               |                                      |   |                       |\n+--------+-----|------------------+      +--------+---|---|-----------------------|-----+\n|        | +---|---+    +-------+ |      |        | +-|---|-+    +-------+    +---|---+ |\n| Region | | Free  | -\u003e | Block | | ---\u003e | Region | | Free  | -\u003e | Block | -\u003e | Free  | |\n|        | +-------+    +-------+ |      |        | +-------+    +-------+    +-------+ |\n+--------+------------------------+      ---------+-------------------------------------+\n```\n\n### [Allocator](./src/allocator.rs)\n\nAnd finally, to put it all together, the allocator is an array of buckets:\n\n```text\n                                 Next Free Block                    Next Free Block\n                      +------------------------------------+   +-----------------------+\n                      |                                    |   |                       |\n     +--------+-------|----------------+      +--------+---|---|-----------------------|-----+\n     |        | +-----|-+    +-------+ |      |        | +-|---|-+    +-------+    +---|---+ |\n0 -\u003e | Region | | Free  | -\u003e | Block | | ---\u003e | Region | | Free  | -\u003e | Block | -\u003e | Free  | |\n     |        | +-------+    +-------+ |      |        | +-------+    +-------+    +-------+ |\n     +--------+------------------------+      +--------+-------------------------------------+\n\n                                              Next Free Block\n                                 +----------------------------------------+\n                                 |                                        |\n     +--------+------------------|-----+      +--------+------------------|------------------+\n     |        | +-------+    +---|---+ |      |        | +-------+    +---|---+    +-------+ |\n1 -\u003e | Region | | Block | -\u003e | Free  | | ---\u003e | Region | | Block | -\u003e | Free  | -\u003e | Block | |\n     |        | +-------+    +-------+ |      |        | +-------+    +-------+    +-------+ |\n     +--------+------------------------+      +--------+-------------------------------------+\n\n..............................................................................................\n\n                                        Next Free Block\n                                 +---------------------------+\n                                 |                           |\n     +--------+------------------|-----+      +--------+-----|-------------------------------+\n     |        | +-------+    +---|---+ |      |        | +---|---+    +-------+    +-------+ |\nN -\u003e | Region | | Block | -\u003e | Free  | | ---\u003e | Region | | Free  | -\u003e | Block | -\u003e | Block | |\n     |        | +-------+    +-------+ |      |        | +-------+    +-------+    +-------+ |\n     +--------+------------------------+      +--------+-------------------------------------+\n```\n\nAll these data structures are a little bit more complicated than that, but\nyou'll have to read the source code for further details.\n\n## TODO\n\n- [ ] Multithreading. Right now the allocator stands behind a mutex, which is\nnot the most efficient way of handling multiple threads. We also need better\ntests, maybe use [`loom`](https://docs.rs/loom/latest/loom/)? See\n[`src/allocator.rs`](./src/allocator.rs) for optimization ideas and low level\ndetails. Here's a quick summary:\n  1. One mutex per bucket instead of one global mutex.\n  2. Fixed number of allocators and load distribution between threads.\n  3. One allocator per thread.\n\n- [ ] Reduce number of system calls for allocations and deallocations. The\nallocator requests just enough memory from the OS to store the user content plus\nheaders, but that could result in calling `mmap` (or equivalent on other\nplatforms) too many times. In order to avoid this, we could figure out how many\nextra pages to request *in general*, for example if we need 1 page to store our\nstuff plus user's stuff, we can request 10 pages instead and avoid kernel code\nexecution for further allocations.\n\n- [ ] Study if it is possible to reduce the size of our headers. On 64 bit\nmachines the size of each block header is 40 bytes and the size of each region\nheader is 48 bytes. Region headers are not so important, but block headers\nare important for small allocations.\n\n- [ ] Memory alignment optimization. The current implementation adds padding\nto the beginning of the block content in order to align pointers. This is only\ndone when the alignment constraint is greater than the pointer size on the\ncurrent machine, and it's not an issue for *small* alignments such as 16 or 32,\nbut it might waste too much space for alignments of 512 and beyond. One possible\nstarting point for this issue: memory regions are already aligned to page size\n(usually 4096), so we could probably take advantage of that somehow.\n\n- [ ] Free blocks searching algorithm optimization. We do have a free list,\nwhich makes finding free blocks easier, and we also have buckets of fixed sizes,\nso this isn't particularly *unoptimized*. Except for sequential large\nallocations of different sizes. Whenever we receive an allocation request that\nwe can't fit in any bucket because it's too large, we use the *Dynamic Bucket*,\nwhich is just a fancy name for a bucket that doesn't care about sizes and can\nstore blocks of 1KB mixed with blocks of 1GB. For that bucket at least, we could\nimplement a heap instead of a normal linked list for searching free blocks, or\nmaybe just store a pointer to the biggest block so that we know immediately if\nwe need to bother the kernel or not.\n\n- [ ] What should we do when `mmap` or `VirtualAlloc` fails? Panic? The\nallocator user doesn't know anything about the failure because\n[`std::alloc::Allocator`](https://doc.rust-lang.org/std/alloc/trait.Allocator.html)\ndoesn't have a return type for deallocations. Panicking doesn't seem the way to\ngo because the program can continue normally, but how and when are we returning\nthe memory region to the kernel then?\n\n- [ ] Remove\n[`std::alloc::GlobalAlloc`](https://doc.rust-lang.org/stable/std/alloc/trait.GlobalAlloc.html)\nimplementation when\n[`std::alloc::Allocator`](https://doc.rust-lang.org/std/alloc/trait.Allocator.html)\nbecomes stable.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantoniosarosi%2Frulloc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantoniosarosi%2Frulloc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantoniosarosi%2Frulloc/lists"}