{"id":20725552,"url":"https://github.com/oneapi-src/unified-memory-framework","last_synced_at":"2025-05-16T10:07:53.181Z","repository":{"id":201175666,"uuid":"696933770","full_name":"oneapi-src/unified-memory-framework","owner":"oneapi-src","description":"A library for constructing allocators and memory pools. It also contains broadly useful abstractions and utilities for memory management. UMF allows users to manage multiple memory pools characterized by different attributes, allowing certain allocation types to be isolated from others and allocated using different hardware resources as required.","archived":false,"fork":false,"pushed_at":"2025-05-12T12:40:53.000Z","size":5205,"stargazers_count":58,"open_issues_count":90,"forks_count":35,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-05-12T13:22:50.326Z","etag":null,"topics":["allocators","cxl","cxl-mem","jemalloc","levelzero","malloc","memory","oneapi","tbbmalloc"],"latest_commit_sha":null,"homepage":"https://oneapi-src.github.io/unified-memory-framework/","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oneapi-src.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.TXT","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"security.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-09-26T17:58:49.000Z","updated_at":"2025-05-12T07:46:52.000Z","dependencies_parsed_at":"2024-04-15T12:47:56.240Z","dependency_job_id":"c1053c18-4281-4a8c-996c-688a8f411f66","html_url":"https://github.com/oneapi-src/unified-memory-framework","commit_stats":null,"previous_names":["oneapi-src/unified-memory-framework"],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Funified-memory-framework","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Funified-memory-framework/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Funified-memory-framework/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Funified-memory-framework/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oneapi-src","download_url":"https://codeload.github.com/oneapi-src/unified-memory-framework/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254509476,"owners_count":22082891,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["allocators","cxl","cxl-mem","jemalloc","levelzero","malloc","memory","oneapi","tbbmalloc"],"created_at":"2024-11-17T04:19:20.269Z","updated_at":"2025-05-16T10:07:48.164Z","avatar_url":"https://github.com/oneapi-src.png","language":"C","readme":"# Unified Memory Framework\n\n[![PR/push](https://github.com/oneapi-src/unified-memory-framework/actions/workflows/pr_push.yml/badge.svg?branch=main\u0026event=push)](https://github.com/oneapi-src/unified-memory-framework/actions/workflows/pr_push.yml)\n[![Coverage](https://gist.githubusercontent.com/bb-ur/3f66c77d7035df39aa75dda8a2ac75b3/raw/umf_coverage_badge.svg)](https://github.com/oneapi-src/unified-memory-framework/actions/workflows/pr_push.yml?query=branch%3Amain)\n[![GitHubPages](https://github.com/oneapi-src/unified-memory-framework/actions/workflows/docs.yml/badge.svg?branch=main)](https://github.com/oneapi-src/unified-memory-framework/actions/workflows/docs.yml)\n[![Nightly](https://github.com/oneapi-src/unified-memory-framework/actions/workflows/nightly.yml/badge.svg?branch=main)](https://github.com/oneapi-src/unified-memory-framework/actions/workflows/nightly.yml)\n[![Coverity build](https://github.com/oneapi-src/unified-memory-framework/actions/workflows/coverity.yml/badge.svg?branch=main)](https://github.com/oneapi-src/unified-memory-framework/actions/workflows/coverity.yml)\n[![Coverity report](https://scan.coverity.com/projects/29761/badge.svg?flat=0)](https://scan.coverity.com/projects/oneapi-src-unified-memory-framework)\n[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/oneapi-src/unified-memory-framework/badge)](https://securityscorecards.dev/viewer/?uri=github.com/oneapi-src/unified-memory-framework)\n\n## Introduction\n\nThe Unified Memory Framework (UMF) is a library for constructing allocators and memory pools. It also contains broadly useful abstractions and utilities for memory management. UMF allows users to manage multiple memory pools characterized by different attributes, allowing certain allocation types to be isolated from others and allocated using different hardware resources as required.\n\n## Usage\n\nFor a quick introduction to UMF usage, please see\n[examples](https://oneapi-src.github.io/unified-memory-framework/examples.html)\ndocumentation, which includes the code of the\n[basic example](https://github.com/oneapi-src/unified-memory-framework/blob/main/examples/basic/basic.c).\nThe are also more advanced that allocates USM memory from the\n[Level Zero device](https://github.com/oneapi-src/unified-memory-framework/blob/main/examples/level_zero_shared_memory/level_zero_shared_memory.c)\nusing the Level Zero API and UMF Level Zero memory provider and [CUDA device](https://github.com/oneapi-src/unified-memory-framework/blob/main/examples/cuda_shared_memory/cuda_shared_memory.c)\nusing the CUDA API and UMF CUDA memory provider.\n\n## Build\n\n### Requirements\n\nRequired packages:\n\n- libhwloc-dev \u003e= 2.3.0 (Linux) / hwloc \u003e= 2.3.0 (Windows)\n- C compiler\n- [CMake](https://cmake.org/) \u003e= 3.14.0\n\nFor development and contributions:\n\n- clang-format-15.0 (can be installed with `python -m pip install clang-format==15.0.7`)\n- cmake-format-0.6 (can be installed with `python -m pip install cmake-format==0.6.13`)\n- black (can be installed with `python -m pip install black==24.3.0`)\n\nFor building tests and multithreaded benchmarks:\n\n- C++ compiler with C++17 support\n\nFor Level Zero memory provider tests:\n\n- Level Zero headers and libraries\n- compatible GPU with installed driver\n\n### Linux\n\nExecutable and binaries will be in **build/bin**.\nThe `{build_config}` can be either `Debug` or `Release`.\n\n```bash\ncmake -B build -DCMAKE_BUILD_TYPE={build_config}\ncmake --build build -j $(nproc)\n```\n\n### Windows\n\nGenerating Visual Studio Project. EXE and binaries will be in **build/bin/{build_config}**.\nThe `{build_config}` can be either `Debug` or `Release`.\n\n```bash\ncmake -B build -G \"Visual Studio 15 2017 Win64\"\ncmake --build build --config {build_config} -j $Env:NUMBER_OF_PROCESSORS\n```\n\n### Benchmark\n\nUMF comes with a single-threaded micro benchmark based on [ubench](https://github.com/sheredom/ubench.h).\nIn order to build the benchmark, the `UMF_BUILD_BENCHMARKS` CMake configuration flag has to be turned `ON`.\n\nUMF also provides multithreaded benchmarks that can be enabled by setting both\n`UMF_BUILD_BENCHMARKS` and `UMF_BUILD_BENCHMARKS_MT` CMake\nconfiguration flags to `ON`. Multithreaded benchmarks require a C++ support.\n\nThe Scalable Pool requirements can be found in the relevant 'Memory Pool\nmanagers' section below.\n\n### Sanitizers\n\nList of sanitizers available on Linux:\n\n- AddressSanitizer\n- UndefinedBehaviorSanitizer\n- ThreadSanitizer\n  - Is mutually exclusive with other sanitizers.\n- MemorySanitizer\n  - Requires linking against MSan-instrumented libraries to prevent false positive reports. More information [here](https://github.com/google/sanitizers/wiki/MemorySanitizerLibcxxHowTo).\n\nList of sanitizers available on Windows:\n\n- AddressSanitizer\n\nListed sanitizers can be enabled with appropriate [CMake options](#cmake-standard-options).\n\n### CMake standard options\n\nList of options provided by CMake:\n\n| Name | Description | Values | Default |\n| - | - | - | - |\n| UMF_BUILD_SHARED_LIBRARY | Build UMF as shared library | ON/OFF | OFF |\n| UMF_BUILD_LEVEL_ZERO_PROVIDER | Build Level Zero memory provider | ON/OFF | ON |\n| UMF_BUILD_CUDA_PROVIDER | Build CUDA memory provider | ON/OFF | ON |\n| UMF_BUILD_LIBUMF_POOL_JEMALLOC | Build the libumf_pool_jemalloc static library | ON/OFF | OFF |\n| UMF_BUILD_TESTS | Build UMF tests | ON/OFF | ON |\n| UMF_BUILD_GPU_TESTS | Build UMF GPU tests | ON/OFF | OFF |\n| UMF_BUILD_BENCHMARKS | Build UMF benchmarks | ON/OFF | OFF |\n| UMF_BUILD_EXAMPLES | Build UMF examples | ON/OFF | ON |\n| UMF_BUILD_FUZZTESTS | Build UMF fuzz tests | ON/OFF | OFF |\n| UMF_BUILD_GPU_EXAMPLES | Build UMF GPU examples | ON/OFF | OFF |\n| UMF_DEVELOPER_MODE | Enable additional developer checks | ON/OFF | OFF |\n| UMF_FORMAT_CODE_STYLE | Add clang, cmake, and black -format-check and -format-apply targets to make | ON/OFF | OFF |\n| UMF_TESTS_FAIL_ON_SKIP | Treat skips in tests as fail | ON/OFF | OFF |\n| UMF_USE_ASAN | Enable AddressSanitizer checks | ON/OFF | OFF |\n| UMF_USE_UBSAN | Enable UndefinedBehaviorSanitizer checks | ON/OFF | OFF |\n| UMF_USE_TSAN | Enable ThreadSanitizer checks | ON/OFF | OFF |\n| UMF_USE_MSAN | Enable MemorySanitizer checks | ON/OFF | OFF |\n| UMF_USE_VALGRIND | Enable Valgrind instrumentation | ON/OFF | OFF |\n| UMF_USE_COVERAGE | Build with coverage enabled (Linux only) | ON/OFF | OFF |\n| UMF_LINK_HWLOC_STATICALLY | Link UMF with HWLOC library statically (proxy library will be disabled on Windows+Debug build) | ON/OFF | OFF |\n| UMF_DISABLE_HWLOC | Disable features that requires hwloc (OS provider, memory targets, topology discovery) | ON/OFF | OFF |\n\n## Architecture: memory pools and providers\n\nA UMF memory pool is a combination of a pool allocator and a memory provider. A memory provider is responsible for\ncoarse-grained memory allocations and management of memory pages, while the pool allocator controls memory pooling\nand handles fine-grained memory allocations.\n\nPool allocator can leverage existing allocators (e.g. jemalloc or tbbmalloc) or be written from scratch.\n\nUMF comes with predefined pool allocators (see [`include/umf/pools`](include/umf/pools)) and providers\n(see [`include/umf/providers`](include/umf/providers)). UMF can also work with user-defined pools and\nproviders that implement a specific interface (see [`include/umf/memory_pool_ops.h`](include/umf/memory_pool_ops.h)\nand [`include/umf/memory_provider_ops.h`](include/umf/memory_provider_ops.h)).\n\nMore detailed documentation is available here: \u003chttps://oneapi-src.github.io/unified-memory-framework/\u003e\n\n### Memory providers\n\n#### Fixed memory provider\n\nA memory provider that can provide memory from a given pre-allocated buffer.\n\n#### OS memory provider\n\nA memory provider that provides memory from an operating system.\n\nOS memory provider supports two types of memory mappings (set by the `visibility` parameter):\n\n1) private memory mapping (`UMF_MEM_MAP_PRIVATE`)\n2) shared memory mapping (`UMF_MEM_MAP_SHARED` - supported on Linux only yet)\n\nIPC API requires the `UMF_MEM_MAP_SHARED` memory `visibility` mode\n(`UMF_RESULT_ERROR_INVALID_ARGUMENT` is returned otherwise).\n\nIPC API uses file descriptor duplication, which requires the `pidfd_getfd(2)` system call to obtain\na duplicate of another process's file descriptor. This system call is supported since Linux 5.6.\nRequired permission (\"restricted ptrace\") is governed by the `PTRACE_MODE_ATTACH_REALCREDS` check\n(see `ptrace(2)`). To allow file descriptor duplication in a binary that opens IPC handle, you can call\n`prctl(PR_SET_PTRACER, ...)` in the producer binary that gets the IPC handle.\nAlternatively you can change the `ptrace_scope` globally in the system, e.g.:\n\n```sh\nsudo bash -c \"echo 0 \u003e /proc/sys/kernel/yama/ptrace_scope\"\n```\n\nThere are available two mechanisms for the shared memory mapping:\n\n1) a named shared memory object (used if the `shm_name` parameter is not NULL) or\n2) an anonymous file descriptor (used if the `shm_name` parameter is NULL)\n\nThe `shm_name` parameter should be a null-terminated string of up to NAME_MAX (i.e., 255) characters none of which are slashes.\n\nAn anonymous file descriptor for the shared memory mapping will be created using:\n\n1) `memfd_secret()` syscall - (if it is implemented and) if the `UMF_MEM_FD_FUNC` environment variable does not contain the \"memfd_create\" string or\n2) `memfd_create()` syscall - otherwise (and if it is implemented).\n\n##### Requirements\n\nIPC API on Linux requires the `PTRACE_MODE_ATTACH_REALCREDS` permission (see `ptrace(2)`)\nto duplicate another process's file descriptor (see above).\n\nPackages required for tests (Linux-only yet):\n\n- libnuma-dev\n\n#### Level Zero memory provider\n\nA memory provider that provides memory from L0 device.\n\nIPC API uses file descriptor duplication, which requires the `pidfd_getfd(2)` system call to obtain\na duplicate of another process's file descriptor. This system call is supported since Linux 5.6.\nRequired permission (\"restricted ptrace\") is governed by the `PTRACE_MODE_ATTACH_REALCREDS` check\n(see `ptrace(2)`). To allow file descriptor duplication in a binary that opens IPC handle, you can call\n`prctl(PR_SET_PTRACER, ...)` in the producer binary that gets the IPC handle.\nAlternatively you can change the `ptrace_scope` globally in the system, e.g.:\n\n```sh\nsudo bash -c \"echo 0 \u003e /proc/sys/kernel/yama/ptrace_scope\"\n```\n##### Requirements\n\n1) Linux or Windows OS\n2) The `UMF_BUILD_LEVEL_ZERO_PROVIDER` option turned `ON` (by default)\n3) IPC API on Linux requires the `PTRACE_MODE_ATTACH_REALCREDS` permission (see `ptrace(2)`)\n   to duplicate another process's file descriptor (see above).\n\nAdditionally, required for tests:\n\n4) The `UMF_BUILD_GPU_TESTS` option turned `ON`\n5) System with Level Zero compatible GPU\n6) Required packages:\n   - liblevel-zero-dev (Linux) or level-zero-sdk (Windows)\n\n#### DevDax memory provider (Linux only)\n\nA memory provider that provides memory from a device DAX (a character device file like `/dev/daxX.Y`).\nIt can be used when large memory mappings are needed.\n\n##### Requirements\n\n1) Linux OS\n2) A character device file /dev/daxX.Y created in the OS.\n\n#### File memory provider (Linux only yet)\n\nA memory provider that provides memory by mapping a regular, extendable file.\n\nIPC API requires the `UMF_MEM_MAP_SHARED` memory `visibility` mode\n(`UMF_RESULT_ERROR_INVALID_ARGUMENT` is returned otherwise).\n\nThe memory visibility mode parameter must be set to `UMF_MEM_MAP_SHARED` in case of FSDAX.\n\n##### Requirements\n\n1) Linux OS\n2) A length of a path of a file to be mapped can be `PATH_MAX` (4096) characters at most.\n\n#### CUDA memory provider\n\nA memory provider that provides memory from CUDA device.\n\n##### Requirements\n\n1) Linux or Windows OS\n2) The `UMF_BUILD_CUDA_PROVIDER` option turned `ON` (by default)\n\nAdditionally, required for tests:\n\n3) The `UMF_BUILD_GPU_TESTS` option turned `ON`\n4) System with CUDA compatible GPU\n5) Required packages:\n   - nvidia-cuda-dev (Linux) or cuda-sdk (Windows)\n\n### Memory pool managers\n\n#### Proxy pool (part of libumf)\n\nThis memory pool is distributed as part of libumf. It forwards all requests to the underlying\nmemory provider. Currently umfPoolRealloc, umfPoolCalloc and umfPoolMallocUsableSize functions\nare not supported by the proxy pool.\n\n#### Disjoint pool (part of libumf)\n\nThe Disjoint pool is designed to keep internal metadata separate from user data.\nThis separation is particularly useful when user data needs to be placed in memory with relatively high latency,\nsuch as GPU memory or disk storage.\n\n#### Jemalloc pool\n\nJemalloc pool is a [jemalloc](https://github.com/jemalloc/jemalloc)-based memory\npool manager built as a separate static library: libjemalloc_pool.a on Linux and\njemalloc_pool.lib on Windows.\nThe `UMF_BUILD_LIBUMF_POOL_JEMALLOC` option has to be turned `ON` to build this library.\n\n[jemalloc](https://github.com/jemalloc/jemalloc) is required to build the jemalloc pool.\n\nIn case of Linux OS jemalloc is built from the (fetched) sources with the following\nnon-default options enabled:\n\n- `--with-jemalloc-prefix=je_` - adds the `je_` prefix to all public APIs,\n- `--disable-cxx` - disables C++ integration, it will cause the `new` and the `delete`\n                    operators implementations to be omitted.\n- `--disable-initial-exec-tls` - disables the initial-exec TLS model for jemalloc's\n                    internal thread-local storage (on those platforms that support\n                    explicit settings), it can allow jemalloc to be dynamically\n                    loaded after program startup (e.g. using `dlopen()`).\n\nThe default jemalloc package is required on Windows.\n\n##### Requirements\n\n1) The `UMF_BUILD_LIBUMF_POOL_JEMALLOC` option turned `ON`\n2) jemalloc is required:\n\n- on Linux and MacOS: jemalloc is fetched and built from sources (a custom build),\n- on Windows: the default jemalloc package is required\n\n#### Scalable Pool (part of libumf)\n\nScalable Pool is a [oneTBB](https://github.com/oneapi-src/oneTBB)-based memory pool manager.\nIt is distributed as part of libumf. To use this pool, TBB must be installed in the system.\n\n##### Requirements\n\nPackages required for using this pool and executing tests/benchmarks (not required for build):\n\n- libtbb-dev (libtbbmalloc.so.2) on Linux or tbb (tbbmalloc.dll) on Windows\n\n### Memspaces (Linux-only)\n\nTODO: Add general information about memspaces.\n\n#### Host all memspace\n\nMemspace backed by all available NUMA nodes discovered on the platform. Can be retrieved\nusing umfMemspaceHostAllGet.\n\n#### Highest capacity memspace\n\nMemspace backed by all available NUMA nodes discovered on the platform sorted by capacity.\nCan be retrieved using umfMemspaceHighestCapacityGet.\n\n#### Highest bandwidth memspace\n\nMemspace backed by an aggregated list of NUMA nodes identified as highest bandwidth after selecting each available NUMA node as the initiator.\nQuerying the bandwidth value requires HMAT support on the platform. Calling `umfMemspaceHighestBandwidthGet()` will return NULL if it's not supported.\n\n#### Lowest latency memspace\n\nMemspace backed by an aggregated list of NUMA nodes identified as lowest latency after selecting each available NUMA node as the initiator.\nQuerying the latency value requires HMAT support on the platform. Calling `umfMemspaceLowestLatencyGet()` will return NULL if it's not supported.\n\n### Proxy library\n\nUMF provides the UMF proxy library (`umf_proxy`) that makes it possible\nto override the default allocator in other programs in both Linux and Windows.\n\nTo enable this feature, the `UMF_BUILD_SHARED_LIBRARY` option needs to be turned `ON`.\n\n#### Linux\n\nIn case of Linux it can be done without any code changes using the `LD_PRELOAD` environment variable:\n\n```sh\nLD_PRELOAD=/usr/lib/libumf_proxy.so myprogram\n```\n\nThe memory used by the proxy memory allocator is mmap'ed:\n\n1) with the `MAP_PRIVATE` flag by default or\n2) with the `MAP_SHARED` flag if the `UMF_PROXY` environment variable contains one of two following strings: `page.disposition=shared-shm` or `page.disposition=shared-fd`. These two options differ in a mechanism used during IPC:\n   - `page.disposition=shared-shm` - IPC uses the named shared memory. An SHM name is generated using the `umf_proxy_lib_shm_pid_$PID` pattern, where `$PID` is the PID of the process. It creates the `/dev/shm/umf_proxy_lib_shm_pid_$PID` file.\n   - `page.disposition=shared-fd` -  IPC API uses file descriptor duplication, which requires the `pidfd_getfd(2)` system call to obtain a duplicate of another process's file descriptor. This system call is supported since Linux 5.6. Required permission (\"restricted ptrace\") is governed by the `PTRACE_MODE_ATTACH_REALCREDS` check (see `ptrace(2)`). To allow file descriptor duplication in a binary that opens IPC handle, you can call `prctl(PR_SET_PTRACER, ...)` in the producer binary that gets the IPC handle. Alternatively you can change the `ptrace_scope` globally in the system, e.g.: `sudo bash -c \"echo 0 \u003e /proc/sys/kernel/yama/ptrace_scope\"`.\n\n**Size threshold**\n\nThe **size threshold** feature (Linux only) causes that all allocations of size less than the given threshold value go to the default system allocator instead of the proxy library.\nIt can be enabled by adding the `size.threshold=\u003cvalue\u003e` string to the `UMF_PROXY` environment variable (with `';'` as a separator), for example: `UMF_PROXY=\"page.disposition=shared-shm;size.threshold=64\"`.\n\n**Remark:** changing a size of allocation (using `realloc()` ) does not change the allocator (`realloc(malloc(threshold - 1), threshold + 1)` still belongs to the default system allocator and `realloc(malloc(threshold + 1), threshold - 1)` still belongs to the proxy library pool allocator).\n\n#### Windows\n\nIn case of Windows it requires:\n\n1) explicitly linking your program dynamically with the `umf_proxy.dll` library\n2) (C++ code only) including `proxy_lib_new_delete.h` in a single(!) source file in your project\n   to override also the `new`/`delete` operations.\n\n## Contributions\n\nAll contributions to the UMF project are most welcome! Before submitting\nan issue or a Pull Request, please read [Contribution Guide](./CONTRIBUTING.md).\n\n## Logging\n\nTo enable logging in UMF source files please follow the guide in the\n[web documentation](https://oneapi-src.github.io/unified-memory-framework/introduction.html#logging).\n\n## Notices\n\nThe contents of this repository may have been developed with support from one or more Intel-operated generative artificial intelligence solutions.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneapi-src%2Funified-memory-framework","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foneapi-src%2Funified-memory-framework","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneapi-src%2Funified-memory-framework/lists"}