{"id":13578732,"url":"https://github.com/cloudwego/sonic-rs","last_synced_at":"2026-02-24T13:06:17.619Z","repository":{"id":200796177,"uuid":"671355892","full_name":"cloudwego/sonic-rs","owner":"cloudwego","description":"A fast Rust JSON library based on SIMD.","archived":false,"fork":false,"pushed_at":"2025-04-10T06:31:51.000Z","size":15876,"stargazers_count":591,"open_issues_count":16,"forks_count":41,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-04-13T19:28:56.290Z","etag":null,"topics":["json","rust","serde","simd"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/sonic-rs","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudwego.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-27T06:17:26.000Z","updated_at":"2025-04-13T12:36:06.000Z","dependencies_parsed_at":"2023-10-27T09:32:58.167Z","dependency_job_id":"3ebf9868-fb25-4adc-8de4-1d08a9456319","html_url":"https://github.com/cloudwego/sonic-rs","commit_stats":null,"previous_names":["cloudwego/sonic-rs"],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudwego%2Fsonic-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudwego%2Fsonic-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudwego%2Fsonic-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudwego%2Fsonic-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudwego","download_url":"https://codeload.github.com/cloudwego/sonic-rs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248769693,"owners_count":21158858,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","rust","serde","simd"],"created_at":"2024-08-01T15:01:33.291Z","updated_at":"2026-02-24T13:06:17.567Z","avatar_url":"https://github.com/cloudwego.png","language":"Rust","funding_links":[],"categories":["Rust","Libraries"],"sub_categories":["Encoding"],"readme":"# sonic-rs\n\n[![Crates.io](https://img.shields.io/crates/v/sonic-rs)](https://crates.io/crates/sonic-rs)\n[![Documentation](https://docs.rs/sonic-rs/badge.svg)](https://docs.rs/sonic-rs)\n[![Website](https://img.shields.io/website?up_message=cloudwego\u0026url=https%3A%2F%2Fwww.cloudwego.io%2F)](https://www.cloudwego.io/)\n[![License](https://img.shields.io/crates/l/sonic-rs)](#license)\n[![Build Status][actions-badge]][actions-url]\n\n[actions-badge]: https://github.com/cloudwego/sonic-rs/actions/workflows/ci.yml/badge.svg\n[actions-url]: https://github.com/cloudwego/sonic-rs/actions\n\nEnglish | [中文](README_ZH.md)\n\nA fast Rust JSON library based on SIMD. It has some references to other open-source libraries like [sonic_cpp](https://github.com/bytedance/sonic-cpp), [serde_json](https://github.com/serde-rs/json), [sonic](https://github.com/bytedance/sonic), [simdjson](https://github.com/simdjson/simdjson), [rust-std](https://github.com/rust-lang/rust/tree/master/library/core/src/num) and more.\n\n***For Golang users to use `sonic_rs`, please see [for_Golang_user.md](https://github.com/cloudwego/sonic-rs/blob/main/docs/for_Golang_user.md)***\n\n***For users to migrate from `serde_json` to `sonic_rs`, can see [serdejson_compatibility](https://github.com/cloudwego/sonic-rs/blob/main/docs/serdejson_compatibility.md)***\n\n## Requirements/Notes\n\n1. Faster in x86_64 or aarch64, other architecture is fallback and maybe very slower.\n\n2. ~~Requires Rust nightly version~~ Support Stable Rust now.\n\n3. Please add the compile options `-C target-cpu=native`\n\n4. Should enable `sanitize` feature to avoid false-positive if you are using LLVM-sanitizer in your program. Don't enable this feature in production, since it will cause 30% performance loss in serialize.\n\n## Quick to use sonic-rs\n\nTo ensure that SIMD instruction is used in sonic-rs, you need to add rustflags `-C target-cpu=native` and compile on the host machine. For example, Rust flags can be configured in Cargo [config](.cargo/config.toml).\n\nAdd sonic-rs in `Cargo.toml`\n\n```\n[dependencies]\nsonic-rs = \"0.3\"\n```\n\n## Features\n1. Serde into Rust struct as `serde_json` and `serde`.\n\n2. Parse/Serialize JSON for untyped `sonic_rs::Value`, which can be mutable.\n\n3. Get specific fields from a JSON with the blazing performance.\n\n4. Use JSON as a lazy array or object iterator with the blazing performance.\n\n5. Support `LazyValue`, `Number` and `RawNumber`(just like Golang's `JsonNumber`) in default.\n\n6. The floating parsing precision is as Rust std in default.\n\n\n## Benchmark\n\nThe main optimization in sonic-rs is the use of SIMD. However, we do not use the two-stage SIMD algorithms from `simd-json`. We primarily use SIMD in the following scenarios:\n1. parsing/serialize long JSON strings\n2. parsing the fraction of float number\n3. Getting a specific elem or field from JSON\n4. Skipping white spaces when parsing JSON\n\nMore details about optimization can be found in [performance.md](docs/performance.md).\n\nBenchmarks environment:\n\n```\nArchitecture:        x86_64\nModel name:          Intel(R) Xeon(R) Platinum 8260 CPU @ 2.40GHz\n```\nAArch64 benchmark data can be found in [benchmark_aarch64.md](docs/benchmark_aarch64.md).\n\nBenchmarks:\n\n- Deserialize Struct: Deserialize the JSON into Rust struct. The defined struct and testdata is from [json-benchmark](https://github.com/serde-rs/json-benchmark)\n\n- Deseirlize Untyped: Deseialize the JSON into an untyped document\n\nThe serialize benchmarks work oppositely.\n\nAll deserialized benchmarks enabled UTF-8 validation and enabled `float_roundtrip` in `serde-json` to get sufficient precision as Rust std. \n\n### Deserialize Struct\n\nThe benchmark will parse JSON into a Rust struct, and there are no unknown fields in JSON text. All fields are parsed into struct fields in the JSON. \n\nSonic-rs is faster than simd-json because simd-json (Rust) first parses the JSON into a `tape`, then parses the `tape` into a Rust struct. Sonic-rs directly parses the JSON into a Rust struct, and there are no temporary data structures. The [flamegraph](assets/pngs/) is profiled in the citm_catalog case.\n\n`cargo bench --bench deserialize_struct -- --quiet`\n\n```\ntwitter/sonic_rs::from_slice_unchecked\n                        time:   [694.74 µs 707.83 µs 723.19 µs]\ntwitter/sonic_rs::from_slice\n                        time:   [796.44 µs 827.74 µs 861.30 µs]\ntwitter/simd_json::from_slice\n                        time:   [1.0615 ms 1.0872 ms 1.1153 ms]\ntwitter/serde_json::from_slice\n                        time:   [2.2659 ms 2.2895 ms 2.3167 ms]\ntwitter/serde_json::from_str\n                        time:   [1.3504 ms 1.3842 ms 1.4246 ms]\n\ncitm_catalog/sonic_rs::from_slice_unchecked\n                        time:   [1.2271 ms 1.2467 ms 1.2711 ms]\ncitm_catalog/sonic_rs::from_slice\n                        time:   [1.3344 ms 1.3671 ms 1.4050 ms]\ncitm_catalog/simd_json::from_slice\n                        time:   [2.0648 ms 2.0970 ms 2.1352 ms]\ncitm_catalog/serde_json::from_slice\n                        time:   [2.9391 ms 2.9870 ms 3.0481 ms]\ncitm_catalog/serde_json::from_str\n                        time:   [2.5736 ms 2.6079 ms 2.6518 ms]\n\ncanada/sonic_rs::from_slice_unchecked\n                        time:   [3.7779 ms 3.8059 ms 3.8368 ms]\ncanada/sonic_rs::from_slice\n                        time:   [3.9676 ms 4.0212 ms 4.0906 ms]\ncanada/simd_json::from_slice\n                        time:   [7.9582 ms 8.0932 ms 8.2541 ms]\ncanada/serde_json::from_slice\n                        time:   [9.2184 ms 9.3560 ms 9.5299 ms]\ncanada/serde_json::from_str\n                        time:   [9.0383 ms 9.2563 ms 9.5048 ms]\n```\n\n\n### Deserialize Untyped\n\nThe benchmark will parse JSON into a document. Sonic-rs seems faster for several reasons:\n- There are also no temporary data structures in sonic-rs, as detailed above.\n- Sonic-rs uses a memory arena for the whole document, resulting in fewer memory allocations, better cache-friendliness, and mutability.\n- The JSON object in `sonic_rs::Value` is an array. Sonic-rs does not build a hashmap.\n\n`cargo bench --bench deserialize_value -- --quiet`\n\n```\ntwitter/sonic_rs_dom::from_slice\n                        time:   [550.95 µs 556.10 µs 562.89 µs]\ntwitter/sonic_rs_dom::from_slice_unchecked\n                        time:   [525.97 µs 530.26 µs 536.06 µs]\ntwitter/serde_json::from_slice\n                        time:   [3.7599 ms 3.8009 ms 3.8513 ms]\ntwitter/serde_json::from_str\n                        time:   [2.8618 ms 2.8960 ms 2.9396 ms]\ntwitter/simd_json::slice_to_owned_value\n                        time:   [1.7302 ms 1.7557 ms 1.7881 ms]\ntwitter/simd_json::slice_to_borrowed_value\n                        time:   [1.1870 ms 1.1951 ms 1.2039 ms]\n\ncanada/sonic_rs_dom::from_slice\n                        time:   [4.9060 ms 4.9568 ms 5.0213 ms]\ncanada/sonic_rs_dom::from_slice_unchecked\n                        time:   [4.7858 ms 4.8728 ms 4.9803 ms]\ncanada/serde_json::from_slice\n                        time:   [16.689 ms 16.980 ms 17.335 ms]\ncanada/serde_json::from_str\n                        time:   [16.398 ms 16.640 ms 16.932 ms]\ncanada/simd_json::slice_to_owned_value\n                        time:   [12.627 ms 12.846 ms 13.070 ms]\ncanada/simd_json::slice_to_borrowed_value\n                        time:   [12.030 ms 12.164 ms 12.323 ms]\n\ncitm_catalog/sonic_rs_dom::from_slice\n                        time:   [1.6657 ms 1.6981 ms 1.7341 ms]\ncitm_catalog/sonic_rs_dom::from_slice_unchecked\n                        time:   [1.5109 ms 1.5253 ms 1.5424 ms]\ncitm_catalog/serde_json::from_slice\n                        time:   [8.1618 ms 8.2566 ms 8.3653 ms]\ncitm_catalog/serde_json::from_str\n                        time:   [7.8652 ms 8.0706 ms 8.3074 ms]\ncitm_catalog/simd_json::slice_to_owned_value\n                        time:   [3.9834 ms 4.0325 ms 4.0956 ms]\ncitm_catalog/simd_json::slice_to_borrowed_value\n                        time:   [3.3196 ms 3.3433 ms 3.3689 ms]\n```\n\n### Serialize Untyped\n\n`cargo bench --bench serialize_value  -- --quiet`\n\nWe serialize the document into a string. In the following benchmarks, sonic-rs appears faster for the `twitter` JSON. The `twitter` JSON contains many long JSON strings, which fit well with sonic-rs's SIMD optimization.\n\n```\ntwitter/sonic_rs::to_string\n                        time:   [380.90 µs 390.00 µs 400.38 µs]\ntwitter/serde_json::to_string\n                        time:   [788.98 µs 797.34 µs 807.69 µs]\ntwitter/simd_json::to_string\n                        time:   [965.66 µs 981.14 µs 998.08 µs]\n\ncitm_catalog/sonic_rs::to_string\n                        time:   [805.85 µs 821.99 µs 841.06 µs]\ncitm_catalog/serde_json::to_string\n                        time:   [1.8299 ms 1.8880 ms 1.9498 ms]\ncitm_catalog/simd_json::to_string\n                        time:   [1.7356 ms 1.7636 ms 1.7972 ms]\n\ncanada/sonic_rs::to_string\n                        time:   [6.5808 ms 6.7082 ms 6.8570 ms]\ncanada/serde_json::to_string\n                        time:   [6.4800 ms 6.5747 ms 6.6893 ms]\ncanada/simd_json::to_string\n                        time:   [7.3751 ms 7.5690 ms 7.7944 ms]\n```\n\n### Serialize Struct\n`cargo bench --bench serialize_struct  -- --quiet`\n\nThe explanation is as mentioned above.\n\n```\ntwitter/sonic_rs::to_string\n                        time:   [434.03 µs 448.25 µs 463.97 µs]\ntwitter/simd_json::to_string\n                        time:   [506.21 µs 515.54 µs 526.35 µs]\ntwitter/serde_json::to_string\n                        time:   [719.70 µs 739.97 µs 762.69 µs]\n\ncanada/sonic_rs::to_string\n                        time:   [4.6701 ms 4.7481 ms 4.8404 ms]\ncanada/simd_json::to_string\n                        time:   [5.8072 ms 5.8793 ms 5.9625 ms]\ncanada/serde_json::to_string\n                        time:   [4.5708 ms 4.6281 ms 4.6967 ms]\n\ncitm_catalog/sonic_rs::to_string\n                        time:   [624.86 µs 629.54 µs 634.57 µs]\ncitm_catalog/simd_json::to_string\n                        time:   [624.10 µs 633.55 µs 644.78 µs]\ncitm_catalog/serde_json::to_string\n                        time:   [802.10 µs 814.15 µs 828.10 µs]\n```\n\n### Get from JSON\n\n`cargo bench --bench get_from -- --quiet`\n\nThe benchmark is getting a specific field from the `twitter.json`. \n\n- sonic-rs::get_unchecked_from_str: without validate\n- sonic-rs::get_from_str: with validate\n- gjson::get_from_str: without validate\n\nSonic-rs utilize SIMD to quickly skip unnecessary fields in the unchecked case, thus enhancing the performance.\n\n```\ntwitter/sonic-rs::get_unchecked_from_str\n                        time:   [75.671 µs 76.766 µs 77.894 µs]\ntwitter/sonic-rs::get_from_str\n                        time:   [430.45 µs 434.62 µs 439.43 µs]\ntwitter/gjson::get_from_str\n                        time:   [359.61 µs 363.14 µs 367.19 µs]\n```\n\n## Usage\n\n### Serde into Rust Type\n\nDirectly use the `Deserialize` or `Serialize` trait.\n\n```rs\nuse sonic_rs::{Deserialize, Serialize}; \n// sonic-rs re-exported them from serde\n// or use serde::{Deserialize, Serialize};\n\n#[derive(Serialize, Deserialize)]\nstruct Person {\n    name: String,\n    age: u8,\n    phones: Vec\u003cString\u003e,\n}\n\nfn main() {\n    let data = r#\"{\n  \"name\": \"Xiaoming\",\n  \"age\": 18,\n  \"phones\": [\n    \"+123456\"\n  ]\n}\"#;\n    let p: Person = sonic_rs::from_str(data).unwrap();\n    assert_eq!(p.age, 18);\n    assert_eq!(p.name, \"Xiaoming\");\n    let out = sonic_rs::to_string_pretty(\u0026p).unwrap();\n    assert_eq!(out, data);\n}\n```\n\n### Get a field from JSON\n\nGet a specific field from a JSON with the `pointer` path. The return is a `LazyValue`, which is a wrapper of a raw valid JSON slice. \n\nWe provide the `get` and `get_unchecked` apis. `get_unchecked` apis should be used in valid JSON, otherwise it may return unexpected result.\n\n\n```rs\nuse sonic_rs::JsonValueTrait;\nuse sonic_rs::{get, get_unchecked, pointer};\n\nfn main() {\n    let path = pointer![\"a\", \"b\", \"c\", 1];\n    let json = r#\"\n        {\"u\": 123, \"a\": {\"b\" : {\"c\": [null, \"found\"]}}}\n    \"#;\n    let target = unsafe { get_unchecked(json, \u0026path).unwrap() };\n    assert_eq!(target.as_raw_str(), r#\"\"found\"\"#);\n    assert_eq!(target.as_str().unwrap(), \"found\");\n\n    let target = get(json, \u0026path);\n    assert_eq!(target.as_str().unwrap(), \"found\");\n    assert_eq!(target.unwrap().as_raw_str(), r#\"\"found\"\"#);\n\n    let path = pointer![\"a\", \"b\", \"c\", \"d\"];\n    let json = r#\"\n        {\"u\": 123, \"a\": {\"b\" : {\"c\": [null, \"found\"]}}}\n    \"#;\n    // not found from json\n    let target = get(json, \u0026path);\n    assert!(target.is_err());\n}\n\n```\n\n### Parse and Serialize into untyped Value\n\nParse a JSON into a `sonic_rs::Value`.\n\n```rs\nuse sonic_rs::{from_str, json};\nuse sonic_rs::JsonValueMutTrait;\nuse sonic_rs::{pointer, JsonValueTrait, Value};\n\nfn main() {\n    let json = r#\"{\n        \"name\": \"Xiaoming\",\n        \"obj\": {},\n        \"arr\": [],\n        \"age\": 18,\n        \"address\": {\n            \"city\": \"Beijing\"\n        },\n        \"phones\": [\n            \"+123456\"\n        ]\n    }\"#;\n\n    let mut root: Value = from_str(json).unwrap();\n\n    // get key from value\n    let age = root.get(\"age\").as_i64();\n    assert_eq!(age.unwrap_or_default(), 18);\n\n    // get by index\n    let first = root[\"phones\"][0].as_str().unwrap();\n    assert_eq!(first, \"+123456\");\n\n    // get by pointer\n    let phones = root.pointer(\u0026pointer![\"phones\", 0]);\n    assert_eq!(phones.as_str().unwrap(), \"+123456\");\n\n    // convert to mutable object\n    let obj = root.as_object_mut().unwrap();\n    obj.insert(\u0026\"inserted\", true);\n    assert!(obj.contains_key(\u0026\"inserted\"));\n\n    let mut object = json!({ \"A\": 65, \"B\": 66, \"C\": 67 });\n    *object.get_mut(\"A\").unwrap() = json!({\n        \"code\": 123,\n        \"success\": false,\n        \"payload\": {}\n    });\n\n    let mut val = json!([\"A\", \"B\", \"C\"]);\n    *val.get_mut(2).unwrap() = json!(\"D\");\n\n    // serialize\n    assert_eq!(serde_json::to_string(\u0026val).unwrap(), r#\"[\"A\",\"B\",\"D\"]\"#);\n}\n```\n\n### JSON Iterator\n\nParse an object or array JSON into a lazy iterator.\n\n```rs\nuse bytes::Bytes;\nuse faststr::FastStr;\nuse sonic_rs::JsonValueTrait;\nuse sonic_rs::{to_array_iter, to_object_iter_unchecked};\nfn main() {\n    let json = Bytes::from(r#\"[1, 2, 3, 4, 5, 6]\"#);\n    let iter = to_array_iter(\u0026json);\n    for (i, v) in iter.enumerate() {\n        assert_eq!(i + 1, v.as_u64().unwrap() as usize);\n    }\n\n    let json = Bytes::from(r#\"[1, 2, 3, 4, 5, 6\"#);\n    let iter = to_array_iter(\u0026json);\n    for elem in iter {\n        // do something for each elem\n\n        // deal with errors when invalid json\n        if elem.is_err() {\n            assert_eq!(\n                elem.err().unwrap().to_string(),\n                \"Expected this character to be either a ',' or a ']' while parsing at line 1 column 17\"\n            );\n        }\n    }\n\n    let json = FastStr::from(r#\"{\"a\": null, \"b\":[1, 2, 3]}\"#);\n    let iter = unsafe { to_object_iter_unchecked(\u0026json) };\n    for ret in iter {\n        // deal with errors\n        if ret.is_err() {\n            println!(\"{}\", ret.unwrap_err());\n            return;\n        }\n\n        let (k, v) = ret.unwrap();\n        if k == \"a\" {\n            assert!(v.is_null());\n        } else if k == \"b\" {\n            let iter = to_array_iter(v.as_raw_str());\n            for (i, v) in iter.enumerate() {\n                assert_eq!(i + 1, v.as_u64().unwrap() as usize);\n            }\n        }\n    }\n}\n```\n\n### JSON LazyValue \u0026 Number \u0026 RawNumber\n\nIf we need to parse a JSON value as a raw string, we can use `LazyValue`.\n\nIf we need to parse a JSON number into an untyped type, we can use `Number`.\n\nIf we need to parse a JSON number ***without loss of precision***, we can use `RawNumber`. It likes `encoding/json.Number` in Golang, and can also be parsed from a JSON string.\n\nDetailed examples can be found in [raw_value.rs](examples/raw_value.rs) and [json_number.rs](examples/json_number.rs).\n\n### Error handle\n\nSonic's errors are followed as `serde-json` and have a display around the error position, examples in [handle_error.rs](examples/handle_error.rs).\n\n\n## FAQs\n\n### About UTF-8\n\nBy default, sonic-rs enable the UTF-8 validation, except for `xx_unchecked` APIs.\n\n\n### About floating point precision\n\nBy default, sonic-rs uses floating point precision consistent with the Rust standard library, and there is no need to add an extra `float_roundtrip` feature like `serde-json` to ensure floating point precision.\n\nIf you want to achieve lossless precision when parsing floating-point numbers, such as Golang `encoding/json.Number` and `serde-json arbitrary_precision`, you can use `sonic_rs::RawNumber`.\n\n\n## Acknowledgement\n\nThanks the following open-source libraries. sonic-rs has some references to other open-source libraries like [sonic_cpp](https://github.com/bytedance/sonic-cpp), [serde_json](https://github.com/serde-rs/json), [sonic](https://github.com/bytedance/sonic), [simdjson](https://github.com/simdjson/simdjson), [yyjson](https://github.com/ibireme/yyjson), [rust-std](https://github.com/rust-lang/rust/tree/master/library/core/src/num) and so on.\n\nWe rewrote many SIMD algorithms from sonic-cpp/sonic/simdjson/yyjson for performance. We reused the de/ser codes and modified necessary parts from serde_json to make high compatibility with `serde`. We reused part codes about floating parsing from rust-std to make it more accurate.\n\nReferenced papers:\n1. [Parsing Gigabytes of JSON per Second](https://arxiv.org/abs/1902.08318)\n2. [JSONSki: streaming semi-structured data with bit-parallel fast-forwarding](https://dl.acm.org/doi/10.1145/3503222.3507719)\n\n\n## Contributing\nPlease read [CONTRIBUTING.md](CONTRIBUTING.md) for information on contributing to sonic-rs.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudwego%2Fsonic-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudwego%2Fsonic-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudwego%2Fsonic-rs/lists"}