{"id":13706690,"url":"https://github.com/vimpunk/mio","last_synced_at":"2025-05-15T13:07:54.477Z","repository":{"id":40450708,"uuid":"105261706","full_name":"vimpunk/mio","owner":"vimpunk","description":"Cross-platform C++11 header-only library for memory mapped file IO","archived":false,"fork":false,"pushed_at":"2024-02-11T17:37:47.000Z","size":165,"stargazers_count":1777,"open_issues_count":49,"forks_count":161,"subscribers_count":56,"default_branch":"master","last_synced_at":"2025-04-15T05:32:08.679Z","etag":null,"topics":["cpp","cpp11","cpp14","cross-platform","file-view","fileviewer","header-only","memory-mapped-file","memory-mapping","mmap"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vimpunk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-09-29T10:44:51.000Z","updated_at":"2025-04-13T20:30:41.000Z","dependencies_parsed_at":"2024-01-03T01:20:04.345Z","dependency_job_id":"e5f4faa9-394b-43c5-81ad-4cd5f039f2e7","html_url":"https://github.com/vimpunk/mio","commit_stats":null,"previous_names":["mandreyel/mio"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vimpunk%2Fmio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vimpunk%2Fmio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vimpunk%2Fmio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vimpunk%2Fmio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vimpunk","download_url":"https://codeload.github.com/vimpunk/mio/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254346624,"owners_count":22055808,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","cpp11","cpp14","cross-platform","file-view","fileviewer","header-only","memory-mapped-file","memory-mapping","mmap"],"created_at":"2024-08-02T22:01:05.421Z","updated_at":"2025-05-15T13:07:49.468Z","avatar_url":"https://github.com/vimpunk.png","language":"C++","funding_links":[],"categories":["C++","C/C++程序设计"],"sub_categories":["资源传输下载"],"readme":"# mio\nAn easy to use header-only cross-platform C++11 memory mapping library with an MIT license.\n\nmio has been created with the goal to be easily includable (i.e. no dependencies) in any C++ project that needs memory mapped file IO without the need to pull in Boost.\n\nPlease feel free to open an issue, I'll try to address any concerns as best I can.\n\n### Why?\nBecause memory mapping is the best thing since sliced bread!\n\nMore seriously, the primary motivation for writing this library instead of using Boost.Iostreams, was the lack of support for establishing a memory mapping with an already open file handle/descriptor. This is possible with mio.\n\nFurthermore, Boost.Iostreams' solution requires that the user pick offsets exactly at page boundaries, which is cumbersome and error prone. mio, on the other hand, manages this internally, accepting any offset and finding the nearest page boundary.\n\nAlbeit a minor nitpick, Boost.Iostreams implements memory mapped file IO with a `std::shared_ptr` to provide shared semantics, even if not needed, and the overhead of the heap allocation may be unnecessary and/or unwanted.\nIn mio, there are two classes to cover the two use-cases: one that is move-only (basically a zero-cost abstraction over the system specific mmapping functions), and the other that acts just like its Boost.Iostreams counterpart, with shared semantics.\n\n### How to create a mapping\nNOTE: the file must exist before creating a mapping.\n\nThere are three ways to map a file into memory:\n\n- Using the constructor, which throws a `std::system_error` on failure:\n```c++\nmio::mmap_source mmap(path, offset, size_to_map);\n```\nor you can omit the `offset` and `size_to_map` arguments, in which case the\nentire file is mapped:\n```c++\nmio::mmap_source mmap(path);\n```\n\n- Using the factory function:\n```c++\nstd::error_code error;\nmio::mmap_source mmap = mio::make_mmap_source(path, offset, size_to_map, error);\n```\nor:\n```c++\nmio::mmap_source mmap = mio::make_mmap_source(path, error);\n```\n\n- Using the `map` member function:\n```c++\nstd::error_code error;\nmio::mmap_source mmap;\nmmap.map(path, offset, size_to_map, error);\n```\nor:\n```c++\nmmap.map(path, error);\n```\n**NOTE:** The constructors **require** exceptions to be enabled. If you prefer\nto build your projects with `-fno-exceptions`, you can still use the other ways.\n\nMoreover, in each case, you can provide either some string type for the file's path, or you can use an existing, valid file handle.\n```c++\n#include \u003csys/types.h\u003e\n#include \u003csys/stat.h\u003e\n#include \u003cfcntl.h\u003e\n#include \u003cmio/mmap.hpp\u003e\n// #include \u003cmio/mio.hpp\u003e if using single header\n#include \u003calgorithm\u003e\n\nint main()\n{\n    // NOTE: error handling omitted for brevity.\n    const int fd = open(\"file.txt\", O_RDONLY);\n    mio::mmap_source mmap(fd, 0, mio::map_entire_file);\n    // ...\n}\n```\nHowever, mio does not check whether the provided file descriptor has the same access permissions as the desired mapping, so the mapping may fail. Such errors are reported via the `std::error_code` out parameter that is passed to the mapping function.\n\n**WINDOWS USERS**: This library *does* support the use of wide character types\nfor functions where character strings are expected (e.g. path parameters).\n\n### Example\n\n```c++\n#include \u003cmio/mmap.hpp\u003e\n// #include \u003cmio/mio.hpp\u003e if using single header\n#include \u003csystem_error\u003e // for std::error_code\n#include \u003ccstdio\u003e // for std::printf\n#include \u003ccassert\u003e\n#include \u003calgorithm\u003e\n#include \u003cfstream\u003e\n\nint handle_error(const std::error_code\u0026 error);\nvoid allocate_file(const std::string\u0026 path, const int size);\n\nint main()\n{\n    const auto path = \"file.txt\";\n\n    // NOTE: mio does *not* create the file for you if it doesn't exist! You\n    // must ensure that the file exists before establishing a mapping. It\n    // must also be non-empty. So for illustrative purposes the file is\n    // created now.\n    allocate_file(path, 155);\n\n    // Read-write memory map the whole file by using `map_entire_file` where the\n    // length of the mapping is otherwise expected, with the factory method.\n    std::error_code error;\n    mio::mmap_sink rw_mmap = mio::make_mmap_sink(\n            path, 0, mio::map_entire_file, error);\n    if (error) { return handle_error(error); }\n\n    // You can use any iterator based function.\n    std::fill(rw_mmap.begin(), rw_mmap.end(), 'a');\n\n    // Or manually iterate through the mapped region just as if it were any other \n    // container, and change each byte's value (since this is a read-write mapping).\n    for (auto\u0026 b : rw_mmap) {\n        b += 10;\n    }\n\n    // Or just change one value with the subscript operator.\n    const int answer_index = rw_mmap.size() / 2;\n    rw_mmap[answer_index] = 42;\n\n    // Don't forget to flush changes to disk before unmapping. However, if\n    // `rw_mmap` were to go out of scope at this point, the destructor would also\n    // automatically invoke `sync` before `unmap`.\n    rw_mmap.sync(error);\n    if (error) { return handle_error(error); }\n\n    // We can then remove the mapping, after which rw_mmap will be in a default\n    // constructed state, i.e. this and the above call to `sync` have the same\n    // effect as if the destructor had been invoked.\n    rw_mmap.unmap();\n\n    // Now create the same mapping, but in read-only mode. Note that calling the\n    // overload without the offset and file length parameters maps the entire\n    // file.\n    mio::mmap_source ro_mmap;\n    ro_mmap.map(path, error);\n    if (error) { return handle_error(error); }\n\n    const int the_answer_to_everything = ro_mmap[answer_index];\n    assert(the_answer_to_everything == 42);\n}\n\nint handle_error(const std::error_code\u0026 error)\n{\n    const auto\u0026 errmsg = error.message();\n    std::printf(\"error mapping file: %s, exiting...\\n\", errmsg.c_str());\n    return error.value();\n}\n\nvoid allocate_file(const std::string\u0026 path, const int size)\n{\n    std::ofstream file(path);\n    std::string s(size, '0');\n    file \u003c\u003c s;\n}\n```\n\n`mio::basic_mmap` is move-only, but if multiple copies to the same mapping are needed, use `mio::basic_shared_mmap` which has `std::shared_ptr` semantics and has the same interface as `mio::basic_mmap`.\n```c++\n#include \u003cmio/shared_mmap.hpp\u003e\n\nmio::shared_mmap_source shared_mmap1(\"path\", offset, size_to_map);\nmio::shared_mmap_source shared_mmap2(std::move(mmap1)); // or use operator=\nmio::shared_mmap_source shared_mmap3(std::make_shared\u003cmio::mmap_source\u003e(mmap1)); // or use operator=\nmio::shared_mmap_source shared_mmap4;\nshared_mmap4.map(\"path\", offset, size_to_map, error);\n```\n\nIt's possible to define the type of a byte (which has to be the same width as `char`), though aliases for the most common ones are provided by default:\n```c++\nusing mmap_source = basic_mmap_source\u003cchar\u003e;\nusing ummap_source = basic_mmap_source\u003cunsigned char\u003e;\n\nusing mmap_sink = basic_mmap_sink\u003cchar\u003e;\nusing ummap_sink = basic_mmap_sink\u003cunsigned char\u003e;\n```\nBut it may be useful to define your own types, say when using the new `std::byte` type in C++17:\n```c++\nusing mmap_source = mio::basic_mmap_source\u003cstd::byte\u003e;\nusing mmap_sink = mio::basic_mmap_sink\u003cstd::byte\u003e;\n```\n\nThough generally not needed, since mio maps users requested offsets to page boundaries, you can query the underlying system's page allocation granularity by invoking `mio::page_size()`, which is located in `mio/page.hpp`.\n\n### Single Header File \nMio can be added to your project as a single header file simply by including `\\single_include\\mio\\mio.hpp`. Single header files can be regenerated at any time by running the `amalgamate.py` script within `\\third_party`.  \n```\npython amalgamate.py -c config.json -s ../include\n```\n\n## CMake\nAs a header-only library, mio has no compiled components. Nevertheless, a [CMake](https://cmake.org/overview/) build system is provided to allow easy testing, installation, and subproject composition on many platforms and operating systems.\n\n### Testing\nMio is distributed with a small suite of tests and examples.\nWhen mio is configured as the highest level CMake project, this suite of executables is built by default.\nMio's test executables are integrated with the CMake test driver program, [CTest](https://cmake.org/cmake/help/latest/manual/ctest.1.html).\n\nCMake supports a number of backends for compilation and linking.\n\nTo use a static configuration build tool, such as GNU Make or Ninja:\n\n```sh\ncd \u003cmio source directory\u003e\nmkdir build\ncd build\n\n# Configure the build\ncmake -D CMAKE_BUILD_TYPE=\u003cDebug | Release\u003e \\\n      -G \u003c\"Unix Makefiles\" | \"Ninja\"\u003e ..\n\n# build the tests\n\u003c make | ninja | cmake --build . \u003e\n\n# run the tests\n\u003c make test | ninja test | cmake --build . --target test | ctest \u003e\n```\n\nTo use a dynamic configuration build tool, such as Visual Studio or Xcode:\n\n```sh\ncd \u003cmio source directory\u003e\nmkdir build\ncd build\n\n# Configure the build\ncmake -G \u003c\"Visual Studio 14 2015 Win64\" | \"Xcode\"\u003e ..\n\n# build the tests\ncmake --build . --config \u003cDebug | Release\u003e\n\n# run the tests via ctest...\nctest --build-config \u003cDebug | Release\u003e\n\n# ... or via CMake build tool mode...\ncmake --build . --config \u003cDebug | Release\u003e --target test\n```\n\nOf course the **build** and **test** steps can also be executed via the **all** and **test** targets, respectively, from within the IDE after opening the project file generated during the configuration step.\n\nMio's testing is also configured to operate as a client to the [CDash](https://www.cdash.org/) software quality dashboard application. Please see the [Kitware documentation](https://cmake.org/cmake/help/latest/manual/ctest.1.html#dashboard-client) for more information on this mode of operation.\n\n### Installation\n\nMio's build system provides an installation target and support for downstream consumption via CMake's [`find_package`](https://cmake.org/cmake/help/v3.0/command/find_package.html) intrinsic function.\nCMake allows installation to an arbitrary location, which may be specified by defining `CMAKE_INSTALL_PREFIX` at configure time.\nIn the absense of a user specification, CMake will install mio to conventional location based on the platform operating system.\n\nTo use a static configuration build tool, such as GNU Make or Ninja:\n\n```sh\ncd \u003cmio source directory\u003e\nmkdir build\ncd build\n\n# Configure the build\ncmake [-D CMAKE_INSTALL_PREFIX=\"path/to/installation\"] \\\n      [-D BUILD_TESTING=False]                         \\\n      -D CMAKE_BUILD_TYPE=Release                      \\\n      -G \u003c\"Unix Makefiles\" | \"Ninja\"\u003e ..\n\n# install mio\n\u003cmake install | ninja install | cmake --build . --target install\u003e\n```\n\nTo use a dynamic configuration build tool, such as Visual Studio or Xcode:\n\n```sh\ncd \u003cmio source directory\u003e\nmkdir build\ncd build\n\n# Configure the project\ncmake [-D CMAKE_INSTALL_PREFIX=\"path/to/installation\"] \\\n      [-D BUILD_TESTING=False]                         \\\n      -G \u003c\"Visual Studio 14 2015 Win64\" | \"Xcode\"\u003e ..\n\n# install mio\ncmake --build . --config Release --target install\n```\n\nNote that the last command of the installation sequence may require administrator privileges (e.g. `sudo`) if the installation root directory lies outside your home directory.\n\nThis installation\n+ copies the mio header files to the `include/mio` subdirectory of the installation root\n+ generates and copies several CMake configuration files to the `share/cmake/mio` subdirectory of the installation root\n\nThis latter step allows downstream CMake projects to consume mio via `find_package`, e.g.\n\n```cmake\nfind_package( mio REQUIRED )\ntarget_link_libraries( MyTarget PUBLIC mio::mio )\n```\n\n**WINDOWS USERS**: The `mio::mio` target `#define`s `WIN32_LEAN_AND_MEAN` and `NOMINMAX`. The former ensures the imported surface area of the Win API is minimal, and the latter disables Windows' `min` and `max` macros so they don't intefere with `std::min` and `std::max`. Because *mio* is a header only library, these defintions will leak into downstream CMake builds. If their presence is causing problems with your build then you can use the alternative `mio::mio_full_winapi` target, which adds none of these defintions.\n\nIf mio was installed to a non-conventional location, it may be necessary for downstream projects to specify the mio installation root directory via either\n\n+ the `CMAKE_PREFIX_PATH` configuration option,\n+ the `CMAKE_PREFIX_PATH` environment variable, or\n+ `mio_DIR` environment variable.\n\nPlease see the [Kitware documentation](https://cmake.org/cmake/help/v3.0/command/find_package.html) for more information.\n\nIn addition, mio supports packaged relocatable installations via [CPack](https://cmake.org/cmake/help/latest/manual/cpack.1.html).\nFollowing configuration, from the build directory, invoke cpack as follows to generate a packaged installation:\n\n```sh\ncpack -G \u003cgenerator name\u003e -C Release\n```\n\nThe list of supported generators varies from platform to platform. See the output of `cpack --help` for a complete list of supported generators on your platform.\n\n### Subproject Composition\nTo use mio as a subproject, copy the mio repository to your project's dependencies/externals folder.\nIf your project is version controlled using git, a git submodule or git subtree can be used to syncronize with the updstream repository.\nThe [use](https://services.github.com/on-demand/downloads/submodule-vs-subtree-cheat-sheet/) and [relative advantages](https://andrey.nering.com.br/2016/git-submodules-vs-subtrees/) of these git facilities is beyond the scope of this document, but in brief, each may be established as follows:\n\n```sh\n# via git submodule\ncd \u003cmy project's dependencies directory\u003e\ngit submodule add -b master https://github.com/mandreyel/mio.git\n\n# via git subtree\ncd \u003cmy project's root directory\u003e\ngit subtree add --prefix \u003cpath/to/dependencies\u003e/mio       \\\n    https://github.com/mandreyel/mio.git master --squash\n```\n\nGiven a mio subdirectory in a project, simply add the following lines to your project's to add mio include directories to your target's include path.\n\n```cmake\nadd_subdirectory( path/to/mio/ )\ntarget_link_libraries( MyTarget PUBLIC \u003cmio::mio | mio\u003e )\n```\n\nNote that, as a subproject, mio's tests and examples will not be built and CPack integration is deferred to the host project.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvimpunk%2Fmio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvimpunk%2Fmio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvimpunk%2Fmio/lists"}