{"id":19706307,"url":"https://github.com/llnl/metall","last_synced_at":"2025-04-29T16:33:17.716Z","repository":{"id":40521464,"uuid":"176391250","full_name":"LLNL/metall","owner":"LLNL","description":"Persistent memory allocator for data-centric analytics","archived":false,"fork":false,"pushed_at":"2025-04-28T01:30:04.000Z","size":80653,"stargazers_count":54,"open_issues_count":3,"forks_count":14,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-28T02:30:04.416Z","etag":null,"topics":["allocator","cpp","ecp","exascale-computing","memory-allocator","persistent-memory"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LLNL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-03-19T00:27:40.000Z","updated_at":"2025-01-14T00:51:53.000Z","dependencies_parsed_at":"2023-10-29T21:34:52.360Z","dependency_job_id":"173c55e1-fcc9-4002-9f81-625a63b1b684","html_url":"https://github.com/LLNL/metall","commit_stats":{"total_commits":726,"total_committers":9,"mean_commits":80.66666666666667,"dds":"0.36225895316804413","last_synced_commit":"d40ae14228a4823323f06d050e5d529fdb606d9c"},"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLNL%2Fmetall","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLNL%2Fmetall/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLNL%2Fmetall/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LLNL%2Fmetall/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LLNL","download_url":"https://codeload.github.com/LLNL/metall/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251540507,"owners_count":21605914,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["allocator","cpp","ecp","exascale-computing","memory-allocator","persistent-memory"],"created_at":"2024-11-11T21:35:02.533Z","updated_at":"2025-04-29T16:33:12.695Z","avatar_url":"https://github.com/LLNL.png","language":"C++","readme":"[![CI Test](https://github.com/LLNL/metall/actions/workflows/ci-test.yml/badge.svg?branch=master)](https://github.com/LLNL/metall/actions/workflows/ci-test.yml)\n[![Documentation Status](https://readthedocs.org/projects/metall/badge/?version=latest)](https://metall.readthedocs.io/en/latest/?badge=latest)\n[![Deploy API Doc](https://github.com/LLNL/metall/actions/workflows/deploy-api-doc.yml/badge.svg?branch=master)](https://github.com/LLNL/metall/actions/workflows/deploy-api-doc.yml)\n\nMetall: A Persistent Memory Allocator for Data-Centric Analytics\n===============================================\n\n- A memory allocator enables applications to transparently allocate data into a file system.\n\n  - From applications, Metall looks like a normal heap allocator.\n\n  - Applications can keep their data beyond single process life cycles and reattach the data in succeeding runs.\n\n  - Leverages the memory-mapped file mechanism (i.e., [mmap(2)](https://man7.org/linux/man-pages/man2/mmap.2.html)) to _map_ application data in files to the main memory.\n\n- Employs the API developed by [Boost.Interprocess](https://www.boost.org/doc/libs/1_69_0/doc/html/interprocess.html).\n\n  - Useful for allocating C++ data structures (including STL containers).\n\n- Incorporates state-of-the-art allocation algorithms.\n\n- Provides a space-efficient snapshot/versioning, leveraging the reflink copy mechanism in file systems.\n\n- See details: [Metall overview slides](docs/publications/metall_101.pdf).\n\n# Getting Started\n\nMetall consists of only header files and requires some header files in Boost C++ Libraries.\n\nAll core files exist under\n[metall/include/metall/](https://github.com/LLNL/metall/tree/master/include/metall).\n\n## Required\n\n- Boost C++ Libraries 1.64 or more.\n  - Build is not required; needs only their header files.\n  - To use JSON containers in Metall, Boost C++ Libraries 1.75 or more is required.\n- C++17 compiler\n  - Tested with GCC 8.1 or more; however, 8.3 or more is recommended due to early implementation of the C++ Filesystem library.\n\n## Build\n\nTo build your program with Metall, all you have to do is just setting include paths such as '-I' or CPLUS_INCLUDE_PATH.\n\nFor example,\n\n```bash\n# Download Boost (Boost C++ Libraries 1.64 or more is required)\n# One can skip this step if Boost is already available.\nwget https://boostorg.jfrog.io/artifactory/main/release/1.78.0/source/boost_1_78_0.tar.gz\ntar xvf boost_1_78_0.tar.gz\nexport BOOST_ROOT=$PWD/boost_1_78_0\n\ngit clone https://github.com/LLNL/metall\nexport METALL_INCLUDE=$PWD/metall/include\n\ng++ -std=c++17 your_program.cpp -lstdc++fs -I${BOOST_ROOT} -I${METALL_INCLUDE}\n```\n\n### Unofficial Support For Clang\nClang can be used instead of GCC to build Metall.\nHowever, we haven't tested it intensively.\nAlso, Boost C++ Libraries 1.69 or more may be required\nif one wants to build Metall with Clang + CUDA.\n\n## Package Manager Support\n\n### Metall with Spack\n\nMetall package is also available on [Spack](https://spack.io/).\n\nAs Metall depends on Boost C++ Libraries,\nSpack also installs a proper version of Boost C++ Libraries automatically, if needed.\n\n```bash\n# Install Metall and Boost C++ Libraries\nspack install metall\n\n# Sets environment variables: BOOST_ROOT and METALL_ROOT.\n# Boost C++ Libraries and Metall are installed at the locations, respectively.\nspack load metall\n\n# Build a program that uses Metall\n# Please note that one has to put 'include' at the end of BOOST_ROOT and METALL_ROOT\ng++ -std=c++17 your_program.cpp -lstdc++fs -I${BOOST_ROOT}/include -I${METALL_ROOT}/include\n```\n\n### Metall with Connan\n\nMetall is also available on [Conan](https://conan.io/), thanks to the [DICE](https://github.com/dice-group) research group.\nConan Metall package information is [here](https://conan.io/center/recipes/metall).\n\n\n## Use Metall from Another CMake Project\n\nTo download and/or link Metall package from a CMake project,\nsee example CMake files placed [here](./example/cmake).\n\n# Build Example Programs\n\nMetall repository contains some example programs under [example directory](./example).\nOne can use CMake to build the examples.\nFor more details, see a page\n[here](https://metall.readthedocs.io/en/latest/advanced_build/cmake/).\n\n\n# Documentations\n\n- [Full documentation](https://metall.readthedocs.io/)\n- [API documentation](https://software.llnl.gov/metall/api/)\n\n## Generate API documentation using Doxygen\n\nA Doxygen configuration file is [here](docs/Doxyfile.in).\n\nTo generate API document:\n\n```bash\ncd metall\nmkdir build_doc\ncd build_doc\ndoxygen ../docs/Doxyfile.in\n```\n\n\n# Publication\n\n```\nKeita Iwabuchi, Karim Youssef, Kaushik Velusamy, Maya Gokhale, Roger Pearce,\nMetall: A persistent memory allocator for data-centric analytics,\nParallel Computing, 2022, 102905, ISSN 0167-8191, https://doi.org/10.1016/j.parco.2022.102905.\n```\n\n* [Parallel Computing](https://www.sciencedirect.com/science/article/abs/pii/S0167819122000114) (journal)\n\n* [arXiv](https://arxiv.org/abs/2108.07223) (preprint)\n\n# About\n\n## Contact\n\n- [GitHub Issues](https://github.com/LLNL/metall/issues) is open.\n  \n- Primary contact: [Keita Iwabuchi (LLNL)](https://github.com/KIwabuchi).\n\n## License\n\nMetall is distributed under the terms of both the MIT license and the\nApache License (Version 2.0). Users may choose either license, at their\noption.\n\nAll new contributions must be made under both the MIT and Apache-2.0\nlicenses.\n\nSee [LICENSE-MIT](LICENSE-MIT), [LICENSE-APACHE](LICENSE-APACHE),\n[NOTICE](NOTICE), and [COPYRIGHT](COPYRIGHT) for details.\n\nSPDX-License-Identifier: (Apache-2.0 OR MIT)\n\n\n## Release\n\nLLNL-CODE-768617\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fllnl%2Fmetall","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fllnl%2Fmetall","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fllnl%2Fmetall/lists"}