{"id":13415265,"url":"https://github.com/simdjson/simdjson","last_synced_at":"2026-04-02T22:30:12.934Z","repository":{"id":37206362,"uuid":"126412363","full_name":"simdjson/simdjson","owner":"simdjson","description":"Parsing gigabytes of JSON per second : used by Facebook/Meta Velox, the Node.js runtime, ClickHouse, WatermelonDB, Apache Doris, Milvus, StarRocks","archived":false,"fork":false,"pushed_at":"2025-04-18T22:19:36.000Z","size":70280,"stargazers_count":20101,"open_issues_count":138,"forks_count":1054,"subscribers_count":241,"default_branch":"master","last_synced_at":"2025-04-22T11:42:38.057Z","etag":null,"topics":["aarch64","arm","arm64","avx2","avx512","c-plus-plus","clang","clang-cl","cpp11","gcc-compiler","json","json-parser","json-pointer","loongarch","neon","simd","sse42","vs2019","x64"],"latest_commit_sha":null,"homepage":"https://simdjson.org","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simdjson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-03-23T00:49:18.000Z","updated_at":"2025-04-22T10:49:52.000Z","dependencies_parsed_at":"2023-10-16T11:17:06.712Z","dependency_job_id":"cf2506ae-1a1c-48cb-b97b-ad4e26928a6c","html_url":"https://github.com/simdjson/simdjson","commit_stats":{"total_commits":2554,"total_committers":139,"mean_commits":18.37410071942446,"dds":0.5469851213782302,"last_synced_commit":"3964f3e5d20c2eaf43e97141511c9e2a85a0089c"},"previous_names":["lemire/simdjson"],"tags_count":92,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simdjson%2Fsimdjson","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simdjson%2Fsimdjson/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simdjson%2Fsimdjson/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simdjson%2Fsimdjson/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simdjson","download_url":"https://codeload.github.com/simdjson/simdjson/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252590372,"owners_count":21772934,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aarch64","arm","arm64","avx2","avx512","c-plus-plus","clang","clang-cl","cpp11","gcc-compiler","json","json-parser","json-pointer","loongarch","neon","simd","sse42","vs2019","x64"],"created_at":"2024-07-30T21:00:46.229Z","updated_at":"2026-04-02T22:30:12.919Z","avatar_url":"https://github.com/simdjson.png","language":"C++","readme":"[![][license img]][license] [![][licensemit img]][licensemit]\n\n\n[![Doxygen Documentation](https://img.shields.io/badge/docs-doxygen-green.svg)](https://simdjson.github.io/simdjson/)\n\nsimdjson : Parsing gigabytes of JSON per second\n===============================================\n\n\u003cimg src=\"images/official_logo/logo_noir/SVG/logo_simdjson_noir.svg\" width=\"40%\" style=\"float: right\"\u003e\n\nJSON is everywhere on the Internet. Servers spend a *lot* of time parsing it. We need a fresh\napproach. The simdjson library uses commonly available SIMD instructions and microparallel algorithms\nto parse JSON 4x  faster than RapidJSON and 25x faster than JSON for Modern C++.\n\n* **Fast:** Over 4x faster than commonly used production-grade JSON parsers.\n* **Record Breaking Features:** Minify JSON  at 6 GB/s, validate UTF-8  at 13 GB/s,  NDJSON at 3.5 GB/s.\n* **Easy:** First-class, easy to use and carefully documented APIs.\n* **Strict:** Full JSON and UTF-8 validation, lossless parsing. Performance with no compromises.\n* **Automatic:** Selects a CPU-tailored parser at runtime. No configuration needed.\n* **Reliable:** From memory allocation to error handling, simdjson's design avoids surprises.\n* **Peer Reviewed:** Our research appears in venues like VLDB Journal, Software: Practice and Experience.\n\nThis library is part of the [Awesome Modern C++](https://awesomecpp.com) list.\n\nTable of Contents\n-----------------\n\n* [Real-world usage](#real-world-usage)\n* [Quick Start](#quick-start)\n* [Documentation](#documentation)\n* [Godbolt](#godbolt)\n* [Performance results](#performance-results)\n* [Packages](#packages)\n* [Bindings and Ports of simdjson](#bindings-and-ports-of-simdjson)\n* [About simdjson](#about-simdjson)\n* [Funding](#funding)\n* [Contributing to simdjson](#contributing-to-simdjson)\n* [License](#license)\n\n\nReal-world usage\n----------------\n\n- [Node.js](https://nodejs.org/)\n- [ClickHouse](https://github.com/ClickHouse/ClickHouse)\n- [Meta Velox](https://velox-lib.io)\n- [Google Pax](https://github.com/google/paxml)\n- [milvus](https://github.com/milvus-io/milvus)\n- [QuestDB](https://questdb.io/blog/questdb-release-8-0-3/)\n- [Clang Build Analyzer](https://github.com/aras-p/ClangBuildAnalyzer)\n- [Shopify HeapProfiler](https://github.com/Shopify/heap-profiler)\n- [StarRocks](https://github.com/StarRocks/starrocks)\n- [Microsoft FishStore](https://github.com/microsoft/FishStore)\n- [Intel PCM](https://github.com/intel/pcm)\n- [WatermelonDB](https://github.com/Nozbe/WatermelonDB)\n- [Apache Doris](https://github.com/apache/doris)\n- [Dgraph](https://github.com/dgraph-io/dgraph)\n- [UJRPC](https://github.com/unum-cloud/ujrpc)\n- [fastgltf](https://github.com/spnda/fastgltf)\n- [vast](https://github.com/tenzir/vast)\n- [ada-url](https://github.com/ada-url/ada)\n- [fastgron](https://github.com/adamritter/fastgron)\n- [WasmEdge](https://wasmedge.org)\n- [RonDB](https://github.com/logicalclocks/rondb)\n- [GreptimeDB](https://github.com/GreptimeTeam/greptimedb)\n- [mamba](https://github.com/mamba-org/mamba)\n- [Ladybird Browser](https://ladybird.org)\n- [SereneDB](https://github.com/serenedb/serenedb)\n\n\nIf you are planning to use simdjson in a product, please work from one of our releases.\n\n\n\n\nQuick Start\n-----------\n\nThe simdjson library is easily consumable with a single .h and .cpp file.\n\n0. Prerequisites: `g++` (version 7 or better) or `clang++` (version 6 or better), and a 64-bit\n   system with a command-line shell (e.g., Linux, macOS, freeBSD). We also support programming\n   environments like Visual Studio and Xcode, but different steps are needed. Users of clang++ may need to specify the C++ version (e.g., `c++ -std=c++17`) since clang++ tends to default on C++98.\n1. Pull [simdjson.h](singleheader/simdjson.h) and [simdjson.cpp](singleheader/simdjson.cpp) into a\n   directory, along with the sample file [twitter.json](jsonexamples/twitter.json). You can download them with the `wget` utility:\n\n   ```\n   wget https://raw.githubusercontent.com/simdjson/simdjson/master/singleheader/simdjson.h https://raw.githubusercontent.com/simdjson/simdjson/master/singleheader/simdjson.cpp https://raw.githubusercontent.com/simdjson/simdjson/master/jsonexamples/twitter.json\n   ```\n2. Create `quickstart.cpp`:\n\n```cpp\n#include \u003ciostream\u003e\n#include \"simdjson.h\"\nusing namespace simdjson;\nint main(void) {\n    ondemand::parser parser;\n    padded_string json = padded_string::load(\"twitter.json\");\n    ondemand::document tweets = parser.iterate(json);\n    std::cout \u003c\u003c uint64_t(tweets[\"search_metadata\"][\"count\"]) \u003c\u003c \" results.\" \u003c\u003c std::endl;\n}\n```\n3. `c++ -o quickstart quickstart.cpp simdjson.cpp`\n4. `./quickstart`\n\n  ```\n   100 results.\n  ```\n\n\nDocumentation\n-------------\n\nUsage documentation is available:\n\n* [Basics](doc/basics.md) is an overview of how to use simdjson and its APIs.\n* [Builder](doc/builder.md) is an overview of how to efficiently write JSON strings using simdjson.\n* [Performance](doc/performance.md) shows some more advanced scenarios and how to tune for them.\n* [Implementation Selection](doc/implementation-selection.md) describes runtime CPU detection and\n  how you can work with it.\n* [API](https://simdjson.github.io/simdjson/) contains the automatically generated API documentation.\n* [Compile-Time Parsing](doc/compile_time.md) presents our compile-time parsing function (C++26 only).\n\n\nGodbolt\n-------------\n\nSome users may want to browse code along with the compiled assembly. You want to check out the following lists of examples:\n* [C++26 reflection example](https://godbolt.org/z/K3Px64TqK)\n* [simdjson examples with errors handled through exceptions](https://godbolt.org/z/7G5qE4sr9)\n* [simdjson examples with errors without exceptions](https://godbolt.org/z/e9dWb9E4v)\n\nPerformance results\n-------------------\n\nThe simdjson library uses three-quarters less instructions than state-of-the-art parser [RapidJSON](https://rapidjson.org). To our knowledge, simdjson is the first fully-validating JSON parser\nto run at [gigabytes per second](https://en.wikipedia.org/wiki/Gigabyte) (GB/s) on commodity processors. It can parse millions of JSON documents per second on a single core.\n\nThe following figure represents parsing speed in GB/s for parsing various files\non an Intel Skylake processor (3.4 GHz) using the GNU GCC 10 compiler (with the -O3 flag).\nWe compare against the best and fastest C++ libraries on benchmarks that load and process the data.\nThe simdjson library offers full unicode ([UTF-8](https://en.wikipedia.org/wiki/UTF-8)) validation and exact\nnumber parsing.\n\n\u003cimg src=\"doc/rome.png\" width=\"60%\"\u003e\n\nThe simdjson library offers high speed whether it processes tiny files (e.g., 300 bytes)\nor larger files (e.g., 3MB). The following plot presents parsing\nspeed for [synthetic files over various sizes generated with a script](https://github.com/simdjson/simdjson_experiments_vldb2019/blob/master/experiments/growing/gen.py) on a 3.4 GHz Skylake processor (GNU GCC 9, -O3).\n\n\u003cimg src=\"doc/growing.png\" width=\"60%\"\u003e\n\n[All our experiments are reproducible](https://github.com/simdjson/simdjson_experiments_vldb2019).\n\n\nFor NDJSON files, we can exceed 3 GB/s with [our  multithreaded parsing functions](https://github.com/simdjson/simdjson/blob/master/doc/parse_many.md).\n\n\nPackages\n------------------------------\n[![Packaging status](https://repology.org/badge/vertical-allrepos/simdjson.svg)](https://repology.org/project/simdjson/versions)\n\n\nBindings and Ports of simdjson\n------------------------------\n\nWe distinguish between \"bindings\" (which just wrap the C++ code) and a port to another programming language (which reimplements everything).\n\n- [ZippyJSON](https://github.com/michaeleisel/zippyjson): Swift bindings for the simdjson project.\n- [libpy_simdjson](https://github.com/gerrymanoim/libpy_simdjson/): high-speed Python bindings for simdjson using [libpy](https://github.com/quantopian/libpy).\n- [pysimdjson](https://github.com/TkTech/pysimdjson): Python bindings for the simdjson project.\n- [cysimdjson](https://github.com/TeskaLabs/cysimdjson): high-speed Python bindings for the simdjson project.\n- [simdjson-rs](https://github.com/simd-lite): Rust port.\n- [simdjson-rust](https://github.com/SunDoge/simdjson-rust): Rust wrapper (bindings).\n- [SimdJsonSharp](https://github.com/EgorBo/SimdJsonSharp): C# version for .NET Core (bindings and full port).\n- [simdjson_nodejs](https://github.com/luizperes/simdjson_nodejs): Node.js bindings for the simdjson project.\n- [simdjson_php](https://github.com/crazyxman/simdjson_php): PHP bindings for the simdjson project.\n- [simdjson_ruby](https://github.com/saka1/simdjson_ruby): Ruby bindings for the simdjson project.\n- [fast_jsonparser](https://github.com/anilmaurya/fast_jsonparser): Ruby bindings for the simdjson project.\n- [simdjson-go](https://github.com/minio/simdjson-go): Go port using Golang assembly.\n- [rcppsimdjson](https://github.com/eddelbuettel/rcppsimdjson): R bindings.\n- [simdjson_erlang](https://github.com/ChomperT/simdjson_erlang): erlang bindings.\n- [simdjsone](https://github.com/saleyn/simdjsone): erlang bindings.\n- [lua-simdjson](https://github.com/FourierTransformer/lua-simdjson): lua bindings.\n- [hermes-json](https://hackage.haskell.org/package/hermes-json): haskell bindings.\n- [zimdjson](https://github.com/EzequielRamis/zimdjson): Zig port.\n- [simdjzon](https://github.com/travisstaloch/simdjzon): Zig port.\n- [JSON-Simd](https://github.com/rawleyfowler/JSON-simd): Raku bindings.\n- [JSON::SIMD](https://metacpan.org/pod/JSON::SIMD): Perl bindings; fully-featured JSON module that uses simdjson for decoding.\n- [gemmaJSON](https://github.com/sainttttt/gemmaJSON): Nim JSON parser based on simdjson bindings.\n- [simdjson-java](https://github.com/simdjson/simdjson-java): Java port.\n- [mruby-fast-json](https://github.com/Asmod4n/mruby-fast-json): mruby binding with high API coverage.\n\nAbout simdjson\n--------------\n\nThe simdjson library takes advantage of modern microarchitectures, parallelizing with SIMD vector\ninstructions, reducing branch misprediction, and reducing data dependency to take advantage of each\nCPU's multiple execution cores.\n\nOur default front-end is called On-Demand, and we wrote a paper about it:\n\n- John Keiser, Daniel Lemire, [On-Demand JSON: A Better Way to Parse Documents?](http://arxiv.org/abs/2312.17149), Software: Practice and Experience 54 (6), 2024.\n\nSome people [enjoy reading the first (2019) simdjson paper](https://arxiv.org/abs/1902.08318): A description of the design\nand implementation of simdjson is in our research article:\n- Geoff Langdale, Daniel Lemire, [Parsing Gigabytes of JSON per Second](https://arxiv.org/abs/1902.08318), VLDB Journal 28 (6), 2019.\n\nWe have an in-depth paper focused on the UTF-8 validation:\n\n- John Keiser, Daniel Lemire, [Validating UTF-8 In Less Than One Instruction Per Byte](https://arxiv.org/abs/2010.03090), Software: Practice \u0026 Experience 51 (5), 2021.\n\nWe also have an informal [blog post providing some background and context](https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of-json-per-second/).\n\nFor the video inclined, we had a talk at QCon San Francisco 2019\u003cbr /\u003e\n[![simdjson at QCon San Francisco 2019](http://img.youtube.com/vi/wlvKAT7SZIQ/0.jpg)](http://www.youtube.com/watch?v=wlvKAT7SZIQ)\u003cbr /\u003e\n(It was the best voted talk, we're kinda proud of it.)\n\nWe also had a CppCon 2025 talk. We show how C++26 reflection allows for one-line serialization (to_json(player)) or deserialization—without invasive macros or manual mapping—using nothing but the C++ standard library. Whether you’re a performance junkie or simply interested in the roadmap for the next decade of C++ development, watch our full talk!\n\n[![simdjson at CppCon 2025](http://img.youtube.com/vi/Mcgk3CxHYMs/0.jpg)](http://www.youtube.com/watch?v=Mcgk3CxHYMs)\u003cbr /\u003e\n\n\n\nCiting this work\n-----------------\n\nIf you use simdjson in published research, please cite the software library. A suitable BibTeX entry is:\n\n```bibtex\n@misc{simdjson,\n  title={{The simdjson library: Parsing Gigabytes of JSON per Second}},\n  author={Daniel Lemire and Geoff Langdale and John Keiser and Paul Dreik and Francisco Thiesen and others},\n  year={2019},\n  howpublished={Software library},\n  note={https://github.com/simdjson/simdjson}\n}\n```\n\nFunding\n-------\n\nThe work is supported by the Natural Sciences and Engineering Research Council of Canada under grants\nRGPIN-2017-03910 and RGPIN-2024-03787.\n\n[license]: LICENSE\n[license img]: https://img.shields.io/badge/License-Apache%202-blue.svg\n\n\n[licensemit]: LICENSE-MIT\n[licensemit img]: https://img.shields.io/badge/License-MIT-blue.svg\n\n\nContributing to simdjson\n------------------------\n\nHead over to [CONTRIBUTING.md](CONTRIBUTING.md) for information on contributing to simdjson, and\n[HACKING.md](HACKING.md) for information on source, building, and architecture/design.\n\n\nStars\n------\n\n[![Star History Chart](https://api.star-history.com/svg?repos=simdjson/simdjson\u0026type=Date)](https://www.star-history.com/#simdjson/simdjson\u0026Date)\n\n\nLicense\n-------\n\nThis code is made available under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html) as well as under the MIT License. As a user, you can pick the license you prefer.\n\nUnder Windows, we build some tools using the windows/dirent_portable.h file (which is outside our library code): it is under the liberal (business-friendly) MIT license.\n\nFor compilers that do not support [C++17](https://en.wikipedia.org/wiki/C%2B%2B17), we bundle the string-view library which is published under the [Boost license](http://www.boost.org/LICENSE_1_0.txt). Like the Apache license, the Boost license is a permissive license allowing commercial redistribution.\n\nFor efficient number serialization, we bundle Florian Loitsch's implementation of the Grisu2 algorithm for binary to decimal floating-point numbers. The implementation was slightly modified by JSON for Modern C++ library. Both Florian Loitsch's implementation and JSON for Modern C++ are provided under the MIT license.\n\nFor runtime dispatching, we use some code from the PyTorch project licensed under 3-clause BSD.\n","funding_links":[],"categories":["C++","HarmonyOS","Libraries","Objects","C/C++ 程序设计","JSON","\u003ca name=\"cpp\"\u003e\u003c/a\u003eC++","c-plus-plus","Data Formats","Repos","Language bindings","File Formats"],"sub_categories":["Windows Manager","Saving/Loading Objects, Compositing Packets","网络服务_其他","C/C++"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimdjson%2Fsimdjson","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimdjson%2Fsimdjson","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimdjson%2Fsimdjson/lists"}