{"id":13606115,"url":"https://github.com/cppfastio/fast_io","last_synced_at":"2025-04-12T08:30:31.081Z","repository":{"id":37317406,"uuid":"461276823","full_name":"cppfastio/fast_io","owner":"cppfastio","description":"Significantly faster input/output for C++20","archived":false,"fork":false,"pushed_at":"2024-05-28T17:14:40.000Z","size":6407,"stargazers_count":555,"open_issues_count":27,"forks_count":47,"subscribers_count":17,"default_branch":"master","last_synced_at":"2024-05-29T08:22:25.843Z","etag":null,"topics":["cpp","cxx","header-only","input-output"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cppfastio.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":"support.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-19T18:21:18.000Z","updated_at":"2024-05-30T10:08:48.485Z","dependencies_parsed_at":"2023-11-06T04:24:45.707Z","dependency_job_id":"9f924703-cab2-4f85-afc8-2a14fa970a50","html_url":"https://github.com/cppfastio/fast_io","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cppfastio%2Ffast_io","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cppfastio%2Ffast_io/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cppfastio%2Ffast_io/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cppfastio%2Ffast_io/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cppfastio","download_url":"https://codeload.github.com/cppfastio/fast_io/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248539745,"owners_count":21121227,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","cxx","header-only","input-output"],"created_at":"2024-08-01T19:01:06.222Z","updated_at":"2025-04-12T08:30:26.067Z","avatar_url":"https://github.com/cppfastio.png","language":"C++","readme":"# fast_io\n\nfast_io is a C++20 input/output library that provides [exceptional speed](readme.md#Benchmarks) and is designed to replace the commonly used `\u003ciostream\u003e` and `\u003ccstdio\u003e` libraries. It is a header-only library and is licensed under the [MIT license](license.txt), making it easy to include in any project. However, it requires a C++20 compiler that supports concepts.\n\n*The original repository for fast_io has been archived due to git bloat over time, but older commits can still be found there.*\n*old commits were [here](https://bitbucket.org/ejsvifq_mabmip/fast_io_archive-2022-01-23).*\n\n## Examples\n### Print to `stdout`\n```cpp\n#include \u003cfast_io.h\u003e\n\nint main() {\n\tprint(\"Hello, fast_io world!\\n\");\n}\n```\n### Memory map file\n```cpp\n#include \u003cfast_io.h\u003e\n\nint main() {\n\tfast_io::native_file_loader file_data(\"text.txt\");\n\t// file_data satisfies std::ranges::contiguous_range\n}\n```\n### Other examples\n\nFor other up-to-date examples, please look in the [`examples/`](examples/) folder.\n\nDeprecated examples are [here](https://bitbucket.org/ejsvifq_mabmip/fast_io_deprecated)\nbut they might not work anymore.\n\nThis i18n repo stores the i18n source files.\n\n[https://bitbucket.org/ejsvifq_mabmip/fast_io_i18n](https://bitbucket.org/ejsvifq_mabmip/fast_io_i18n)\n\n## F.A.Q.\n### What does \"fast\" mean in fast_io?\n\nThe term ```fast``` in ```fast_io``` does not necessarily imply that it is the fastest input/output library available (as it would have been named as ```fastest_io``` instead). Instead, the term refers to the fact that ```fast_io``` is notably faster than ```\u003ciostream\u003e``` and ```\u003ccstdio\u003e``` libraries in all cases.\n\n### What are the supported platforms, compilers...?\nSee [support.md](support.md) for a list of supported things and platform specific usage notes.\n\n### Other questions?\nYou can ask questions on the [fast_io discord server](https://discord.gg/wYfh8kk)\nor QQ group: `801441303`.\n\n## Documentation\nSee either https://ewindy.gitee.io/fast_io_rst/index.html or https://gitee.com/qabeowjbtkwb/fast_io/wikis.\n\n## Features\n- As close to system call as possible.\n- Unicode support (UTF-8, UTF-16, UTF-32) + codecvt for GB18030 and UTF-EBCDIC (no libc correctly deals with it)\n- RAII for C `FILE*`, POSIX `fd` and win32/NT `HANDLE`\n- Interop with `\u003ccstdio\u003e` and `\u003ciostream\u003e` \n- No easily misused stuff like `std::endl`\n- Static I/O manipulators instead of format strings.\n- Optional locale.\n- Stateless I/O manipulation.\n- Consistent error handling; when available, exceptions as the **only** error reporting mechanism (no `std::error_code`, `std::system_error` or useless bounds checking)\n- Freestanding mode.\n- Address Sanitizer special code for protecting memory safety issues.\n- Dynamic Instrumentation Support\n\n- Supports POSIX iconv. You can use fast_io for your coding conversion.\n- Binary serialization for trivially copyable types and standard containers\n- Plays well with C++ containers (e.g. `std::vector\u003cfast_io::obuf_file\u003e` is valid)\n- Basic/Lua/Python/etc format (print, scan). No support for C and C++ since they are security hazards.\n- Provide APIs to expose the internal implementation of `FILE*` and C++ streams.\n\n- Native Handle Interface\n- Extremely easy to support custom devices\n- C and C++ style compile time open mode parsing.\n- No `traits_type` and `EOF`\n- Dynamic Type Support\n- Multi Process\n- Memory mapping\n- debugging IO (optionally with GUI)\n- Round-trip floating point algorithm\n- Hash algorithms support: intrinsic SHA-1, intrinsic HMAC-SHA1, intrinsic SHA-256, intrinsic HMAC-SHA256, SHA-512, HMAC-SHA512 as well as non crypto hash algorithms, such as Jenkins Hash.\n- zlib compression/decompression\n- Filesystem\n- OpenSSL BIO, Qt QFile, MFC CFile support\n\n### Post C++20\n  1. Module support\n  2. Coroutine support for async IO\n  3. Improve and refactor code once [Zero-overhead deterministic exceptions](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf) are added to the standard\n\n## Benchmarks\n\n1. I/O 10M integers\n\nGoal: Print out ten million integers from 0 to 10M to file. Then reopen that file to scan back.\n\nAll benchmarks are in benchmarks/0000.10m_size_t/unit.\n\nNotice: I modified libstdc++'s BUFSIZ 1048576 due to BUFSIZ is too small (512 bytes) for MinGW-W64, or it performs horribly.\n\n\n| Platform                       |        Windows          |MinGW-W64 GCC 11.0.0   |   MSVCRT + libstdc++                                 |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n|                                                                                                                                         |\n\n| Method                         |       Output time       |      Input time       |   Comment                                            |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n| stdio.h(fprintf/fscanf)        |      2.412987s          |   5.607791s           |                                                      |\n| fstream                        |      0.462012s          |   1.192s              |                                                      |\n| fstream with rdbuf.sputc trick |      0.33895s           |   1.170173s           |                                                      |\n| fast_io::i/obuf_file           |      0.04903s           |   0.080996s           |                                                      |\n| fast_io::i/obuf_file_mutex     |      0.146064s          |   0.113155s           | thread safe                                          |\n| c_locale_i/obuf_file (\"C\")     |      0.065988s          |   0.086012s           | imbued with locale, locale \"C\"                       |\n| c_locale_i/obuf_file local     |      0.153995s          |   :x:                 | imbued with locale, locale \"\"                        |\n| fmt::format_int+obuf_file      |      0.122999s          |   :x:                 |                                                      |\n| fmt::format_int+ofstream       |      0.209055s          |   :x:                 |                                                      |\n| fmt::format+ofstream           |      0.548s             |   :x:                 | fmt makes things slower                              |\n| fmt::print                     |      0.663996s          |   :x:                 | fmt makes things slower                              |\n| std::to_chars+obuf_file        |      0.12s              |   :x:                 |                                                      |\n| std::to_chars+ofstream         |      0.192s             |   :x:                 |                                                      |\n| fast_io::c_file_unlocked       |      0.098999s          |   0.126003s           | I hacked MSVCRT's FILE* implementation               |\n| fast_io::c_file                |      0.298988s          |   0.318001s           | Thread Safe. I hacked MSVCRT's FILE* implementation |\n| fast_io::filebuf_file          |      0.048999s          |   0.081s              | I hacked libstdc++'s streambuf/filebuf implementation|\n| fast_io::iobuf_utf8_file_char16|      0.124s       |   0.112001s             | UTF-16=\u003eUTF-8 with SSE|\n| fast_io::iobuf_utf8_file_char32|      0.110999s       |   0.111011s             | UTF-32=\u003eUTF-8 with SSE|\n| std::wofstream             |      2.64s       |   3.843735s             | wofstream with std::locale codecvt. Extremely slow tbh.|\n| fast_io::wfilebuf_io_observer  |      2.415692s       |   2.497704s         | wofstream with std::locale codecvt. This proves fstream can never get fixed.|\n| Rust language  |      0.483s       | :x:            |  RUST IS SLOW. Also Rust does not deal with locale. Think how bad it is.  |\n| Rust itoa library 0.4.6 | \u003e 0.165s | :x: | I ignored the `\\n` part for it to ensure no bias. |\n\n\nRust language is 10x slower than fast\\_io. + binary bloat and itoa library is still extremely slow and usable for me. It is at least 3x slower than fast\\_io.\n\n\nRun the same test on MSVC 19.26.28805.\n\n\n| Platform                       |       Windows           |  MSVC 19.26.28805     |  Install fmtlib wastes time of my life               |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n|                                                                                                                                         |\n\n| Method                         |       Output time       |      Input time       |   Comment                                            |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n| stdio.h(fprintf/fscanf)        |      1.5353597s         |   1.4157233s          |                                                      |\n| fstream                        |      3.6350262s         |   3.8420339s          |                                                      |\n| fstream with rdbuf.sputc trick |      3.3735902s         |   3.8145566s          |                                                      |\n| fast_io::i/obuf_file           |      0.0631433s         |   0.1030554s          |                                                      |\n| fast_io::i/obuf_file_mutex     |      0.2190659s         |   0.2485886s          | thread safe                                          |\n| std::to_chars+obuf_file        |      0.1641641s         |   :x:                 |                                                      |\n| std::to_chars+ofstream         |      0.5461922s         |   :x:                 |                                                      |\n| fast_io::c_file_unlocked       |      0.1102575s         |   0.2399757s          | I hacked Universal CRT's FILE* implementation        |\n| fast_io::c_file                |      0.2034755s         |   0.2621148s          | Thread Safe. I hacked UCRT's FILE* implementation    |\n| fast_io::filebuf_file          |      0.126661s          |   0.2378803s          | I hacked MSVC STL's streambuf/filebuf implementation |\n\nRun the same test on GCC 11. glibc + libstdc++\n\n\n| Platform                       |       Linux             |  GCC 11.0.0           |         glibc + libstdc++                            |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n|                                                                                                                                         |\n\n| Method                         |       Output time       |      Input time       |   Comment                                            |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n| stdio.h(fprintf/fscanf)        |      0.532792935s       |   0.591907111s        |                                                      |\n| fstream with rdbuf.sputc trick |      0.318896068s       |   0.429406415s        |                                                      |\n| fast_io::i/obuf_file           |      0.050300857s       |   0.065372395s        |                                                      |\n| fast_io::i/obuf_file_mutex     |      0.05290654s        |   0.083040518s        | thread safe                                          |\n| c_locale_i/obuf_file (\"C\")     |      0.051939052s       |   0.065820056s        | imbued with locale, locale \"C\"                       |\n| c_locale_i/obuf_file local     |      0.162406082s       |   :x:                 | imbued with locale, locale \"\"                        |\n| std::to_chars+obuf_file        |      0.115453587s       |   :x:                 |                                                      |\n| fmt::format_int+obuf_file      |      0.1183587s         |   :x:                 |                                                      |\n| fmt::format_int+ofstream       |      0.195914384s       |   :x:                 |                                                      |\n| fmt::format+ofstream           |      0.633590975s       |   :x:                 | fmt makes things slower                              |\n| fmt::print                     |      0.495270371s       |   :x:                 | fmt makes things slower                              |\n| boost::iostreams               |      0.400906063s       |   0.444717051s        | Using boost iostreams does not make your code faster |\n| fast_io::c_file_unlocked       |      0.060076723s       |   0.073299716s        | I hacked glibc's FILE* implementation                |\n| fast_io::c_file                |      0.092490191s       |   0.104545535s        | Thread Safe. I hacked glibc's FILE* implementation   |\n| fast_io::filebuf_file          |      0.052251608s       |   0.06655806s         | I hacked libstdc++'s streambuf/filebuf implementation|\n\nYou can see fast_io can also boost the performance of existing facilities for 10x! Yes, it can even improve FILE* and fstream's performance for 10x depending on platforms since I use concepts to abstract them all. fmtlib actually slows down I/O performance.\n\n2. Output 10M double in round-trip mode with Ryu algorithm\n\nWe only perform this test for MSVC since only msvc's charconv implements it. Yes. fast_io defeats msvc's charconv for over 20% for running the same algorithm.\n\nAll benchmarks are in benchmarks/0001.10m_double/charconv.\n\nRun the same test on MSVC 19.26.28805.\n\n\n| Platform                       |       Windows           |  MSVC 19.26.28805     |                  |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n|                                                                                                                                         |\n\n| Method                         |       Output time       |                           Comment                                            |\n|--------------------------------|-------------------------|------------------------------------------------------------------------------|\n| i/obuf_file                    |      0.4653818s         |                                                                              |\n| charconv + obuf_file           |      0.6011s            |                                                                              |\n\n3. Raw I/O Performance\n\nAll benchmarks are in benchmarks/0014.file_io/file_io.\n\nOutput 100000000x \"Hello World\\n\"\n\nNotice: I modified libstdc++'s std::filebuf's BUFSIZ to 1048576 due to BUFSIZ is too small (512 bytes) for MinGW-W64 or it performs horribly.\n\n\n\n| Platform                       |        Windows          |MinGW-W64 GCC 11.0.0   |   MSVCRT + libstdc++                                 |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n|                                                                                                                                         |\n\n| Method                         |       Output time       |                           Comment                                            |\n|--------------------------------|-------------------------|------------------------------------------------------------------------------|\n| fwrite                         |      2.524001s          |                                                                              |\n| fstream                        |      1.013001s          |                                                                              |\n| fast_io::obuf_file             |      0.437998s          |                                                                              |\n| fast_io::obuf_file_mutex       |      1.371s             |Thread safe                                                                   |\n| fast_io::c_file_unlocked       |      1.164997s          |I hacked MSVCRT's FILE* implementation                                        |\n| fast_io::c_file                |      3.337945s          |Thread Safe. I hacked MSVCRT's FILE* implementation. Need further optimization|\n| fast_io::filebuf_file          |      0.467001s          |I hacked libstdc++'s std::filebuf implementation                              |\n\n\n| Platform                       |        Linux            |          GCC 11.0.0   |     glibc + libstdc++                                |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n|                                                                                                                                         |\n\n| Method                         |       Output time       |                           Comment                                            |\n|--------------------------------|-------------------------|------------------------------------------------------------------------------|\n| fwrite                         |      1.457288317s       |                                                                              |\n| fstream                        |      1.249783346s       |                                                                              |\n| fast_io::obuf_file             |      0.494827134s       |                                                                              |\n| fast_io::obuf_file_mutex       |      0.497138826s       |Thread safe                                                                   |\n| fast_io::c_file_unlocked       |      0.687976666s       |I hacked glibc's FILE* implementation                                         |\n| fast_io::c_file                |      0.910792697s       |Thread Safe. I hacked glibc's FILE* implementation                            |\n| fast_io::filebuf_file          |      0.526955039s       |I hacked libstdc++'s std::filebuf implementation                              |\n\n\n| Platform                       |        Windows          |  MSVC 19.26.28805     |      UCRT + MSVC STL                                 |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n|                                                                                                                                         |\n\n| Method                         |       Output time       |                           Comment                                            |\n|--------------------------------|-------------------------|------------------------------------------------------------------------------|\n| fwrite                         |      3.3139122s         |                                                                              |\n| fstream                        |      1.7184119s         |                                                                              |\n| fast_io::obuf_file             |      0.7996034s         |                                                                              |\n| fast_io::obuf_file_mutex       |      2.2949221s         |Thread safe. It looks like std::mutex is horribly slow for MSVC STL.          |\n| fast_io::c_file_unlocked       |      1.2103924s         |I hacked UCRT's FILE* implementation                                          |\n| fast_io::c_file                |      2.3604295s         |Thread Safe. I hacked UCRT's FILE* implementation                             |\n| fast_io::filebuf_file          |      1.2805368s         |I hacked MSVC STL's std::filebuf implementation                               |\n\n\n4. Binary Size\nJust use the benchmark in benchmarks/0014.file_io/file_io.\nDude, you should avoid stream as plague tbh. It is not healthy.\n\n\n| Platform                       |        Windows          |MinGW-W64 GCC 11.0.0   |   MSVCRT + libstdc++ + static compile                |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n|                                                                                                                                         |\n\n| Method                         |      Binary Size        |                           Comment                                            |\n|--------------------------------|-------------------------|------------------------------------------------------------------------------|\n| fstream                        |      925KB              |Use fstream is not good for your health since std::locale bloats your binary. |\n| fast_io::obuf_file             |      155KB              |                                                                              |\n| fast_io::c_file_unlocked       |      157KB              |I hacked MSVCRT's FILE* implementation                                        |\n| fast_io::c_file                |      157KB              |Thread Safe. I hacked MSVCRT's FILE* implementation                           |\n| fast_io::filebuf_file          |      933KB              |I hacked libstdc++'s std::filebuf implementation. C++ stream sucks            |\n\n\n5. Code Convert\n\nGenerate 100000000 🚄 emojis by using the program in benchmarks/0020.utf/fill_nc.cc\n\nBenchmarks are in examples/0043.iconv\nUniversal iconv. (UTF-8 to GB18030 as an example)\niconv test:\n| Platform                       |        Windows          |MinGW-W64 GCC 11.0.0   |   MSVCRT + libstdc++                                 |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n|                                                                                                                                         |\n\n| Method                         |       Elapsed time      |                           Comment                                            |\n|--------------------------------|-------------------------|------------------------------------------------------------------------------|\n| iconv command                  |      1.529s             |                                                                              |\n| universal.cc                   |      1.293s             |  use POSIX libiconv                                                        |\n\n\nUTF8-\u003eUTF16LE\n\nBenchmarks are in examples/0022.utf\n\niconv test:\n| Platform                       |        Windows          |MinGW-W64 GCC 11.0.0   |   MSVCRT + libstdc++                                 |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n|                                                                                                                                         |\n\n| Method                         |       Elapsed time      |                           Comment                                            |\n|--------------------------------|-------------------------|------------------------------------------------------------------------------|\n| iconv command                  |      0.967s             |  GNU iconv. No BOM which sucks                                              |\n| utf8_file_to_utf16_file.cc     |      0.498s             |  I use the SSE algorithms provided by the utf-utils project.                |\n\nUTF8-\u003eUTF32LE\n\nBenchmarks are in examples/0022.utf\n\niconv test:\n| Platform                       |        Windows          |MinGW-W64 GCC 11.0.0   |   MSVCRT + libstdc++                                 |\n|--------------------------------|-------------------------|-----------------------|------------------------------------------------------|\n|                                                                                                                                         |\n\n| Method                         |       Elapsed time      |                           Comment                                            |\n|--------------------------------|-------------------------|------------------------------------------------------------------------------|\n| iconv command                  |      0.844s             |  GNU iconv. No BOM which sucks                                           |\n| utf8_file_to_utf32_file.cc     |      0.442s             |  I use the SSE algorithms provided by the utf-utils project.                   |\n\n## Credits\n\nThe creation and development of this project were made possible thanks to the valuable contributions of various open-source projects. While the code was not copied directly from these projects, I used them as references and re-implemented them to suit the specific purposes of this library. In some cases, integration issues arose that required modifications to the original code. I am grateful to these projects and their developers for their commitment to making their code open and accessible to the wider community.\n\n|Project          | Url\n|-----------------|-----------------------------------------\n|Grisu-Exact      |https://github.com/jk-jeon/Grisu-Exact\n|Ryu              |https://github.com/ulfjack/ryu\n|SHA-Intrinsics   |https://github.com/noloader/SHA-Intrinsics\n|SHA1             |https://github.com/vog/sha1\n|UTF-utils        |https://github.com/BobSteagall/utf_utils\n|jenkins-hash-java|https://github.com/vkandy/jenkins-hash-java\n|md5              |https://github.com/JieweiWei/md5\n|ReactOS          |https://github.com/reactos/reactos\n|dirent_h         |https://github.com/win32ports/dirent_h\n|GNU C library    |https://www.gnu.org/software/libc/\n|GNU Newlib       |https://sourceware.org/newlib/\n|Dragonbox        |https://github.com/jk-jeon/dragonbox\n|JEAIII           |https://github.com/jeaiii/itoa\n|Crypto++         |https://github.com/weidai11/cryptopp\n|MyItoA           |https://gitee.com/xjkp2283572185/MyStd\n","funding_links":[],"categories":["Miscellaneous","C++"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcppfastio%2Ffast_io","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcppfastio%2Ffast_io","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcppfastio%2Ffast_io/lists"}