{"id":20797720,"url":"https://github.com/littlefox94/pmalloc","last_synced_at":"2025-07-12T02:09:33.955Z","repository":{"id":77328210,"uuid":"372048002","full_name":"LittleFox94/pmalloc","owner":"LittleFox94","description":"Allocating physical memory in linux userspace","archived":false,"fork":false,"pushed_at":"2023-06-14T10:34:18.000Z","size":38,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-05-06T19:04:32.034Z","etag":null,"topics":["allocator","linux","physical-memory","userspace-driver"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LittleFox94.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["LittleFox94"]}},"created_at":"2021-05-29T18:47:11.000Z","updated_at":"2024-08-01T06:49:34.000Z","dependencies_parsed_at":"2023-09-05T19:30:48.403Z","dependency_job_id":null,"html_url":"https://github.com/LittleFox94/pmalloc","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/LittleFox94/pmalloc","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LittleFox94%2Fpmalloc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LittleFox94%2Fpmalloc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LittleFox94%2Fpmalloc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LittleFox94%2Fpmalloc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LittleFox94","download_url":"https://codeload.github.com/LittleFox94/pmalloc/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LittleFox94%2Fpmalloc/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264923504,"owners_count":23683761,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["allocator","linux","physical-memory","userspace-driver"],"created_at":"2024-11-17T16:35:02.049Z","updated_at":"2025-07-12T02:09:33.919Z","avatar_url":"https://github.com/LittleFox94.png","language":"C","funding_links":["https://github.com/sponsors/LittleFox94"],"categories":[],"sub_categories":[],"readme":"[![.github/workflows/test-and-build.yaml](https://github.com/LittleFox94/pmalloc/actions/workflows/test-and-build.yaml/badge.svg)](https://github.com/LittleFox94/pmalloc/actions/workflows/test-and-build.yaml)\n[![codecov](https://codecov.io/gh/LittleFox94/pmalloc/branch/main/graph/badge.svg?token=TWVR9LA7Z0)](https://codecov.io/gh/LittleFox94/pmalloc)\n\nThis is an allocator for physically continuous memory in linux\nuserspace.\n\n## Implementation details:\n\nMemory is requested from the system with mmap, allocating the smallest hugepage\nsize available on the system matching the request - so on x86\\_64 this means\neither a 4KiB, a 2MiB or a 1GiB page. Memory not used in a page will be used for\nother allocations.\n\nEach block allocated from the system has a header `struct pmem`, storing the\noriginally allocated size (how big the `mmap`'d block is), the amount of memory in\nuse/freed and the physical address of the start of the block, which is retrieved by\nreading `/proc/self/pagemap`, this requires special privileges! Check proc(5) for details.\nThis header also contains pointer to the previous and the next one, forming a\ndouble-linked list of memory blocks. \n\nWhen the user of this library makes an allocation, first every already allocated\nblock is checked if it has enough space for that allocation. Only space after the\nlast allocation will be used, so if you `a = pmalloc(), b=pmalloc(), free(a)`, the\nmemory used by `a` is not usable again - only the space needed for it is added to\nthe `size_freed` of the block. This design decision makes the metadata to store per\nblock very simple and fits the use case I needed this for - allocating some memory\non start of program, maybe allocating some more at some point and freeing at the end.\n\nIf a memory blocks `size_freed` matches the `size_used`, it will be given back to the\nsystem, so removed from the linked list and `munmap`'d.\n\nThe available huge page sizes are retrieved by listing the directories in\n`/sys/kernel/mm/hugepages` (`retrieve_hugepage_sizes` does this). Before doing a `mmap`,\n`pmalloc` will check if enough (mostly 1, if more it might already be problematic) pages\nof the needed size are available by checking `/sys/kernel/mm/hugepages/$dir/free_hugepages`.\nIf there aren't, it tries to increase `nr_hugepages` in the same directory; this might work,\nbut probably won't when the system is online for some time already, since memory\nfragmentation is a thing (check the linux docs for details: admin-guide/mm/hugetlbpage).\n\nThe whole thing is quite chatty on anything unexpected. There is no way to make it\nnon-chatty, deal with it.\n\n\n## Tests are failing!\n\nOne of the tests is allocating a 3MB chunk. On x86\\_64, this needs to allocate a 1GiB\nhuge page. Since those aren't allocated by default, you systems memory might be to\nfragmented to allocate one. There is a linux cmdline flag to allocate some on boot\n(again, check out linux docs on that admin-guide/mm/hugetlbpage). If only the 3MiB test\nis failing, you can consider it as `success`. I might change the test one day to only\nlog that as error when the system would have had the hugepages for that.\n\n\n## What did you need this for?\n\nTesting [LF OS](https://github.com/LittleFox94/lf-os_amd64) drivers on linux in userspace,\nespecially one of the few drivers LF OS has in kernel space: XHCI DbC (debug class).\nUnbinding the device driver from the PCI node and then running the driver code against it,\nusing this library for physical memory allocations.\n\nIt's very cursed, and also kinda funny - since this is one of the few drivers in LF OS\nkernel space, it makes a lot of sense to test it in linux userspace - with linux having\nmost drivers in the kernel :D\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flittlefox94%2Fpmalloc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flittlefox94%2Fpmalloc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flittlefox94%2Fpmalloc/lists"}