{"id":13615941,"url":"https://github.com/lovasoa/bad_json_parsers","last_synced_at":"2025-04-05T22:08:36.205Z","repository":{"id":35957680,"uuid":"107819708","full_name":"lovasoa/bad_json_parsers","owner":"lovasoa","description":"Exposing problems in json parsers of several programming languages.","archived":false,"fork":false,"pushed_at":"2022-12-08T08:56:40.000Z","size":195,"stargazers_count":365,"open_issues_count":9,"forks_count":23,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-03-29T21:06:33.376Z","etag":null,"topics":["json","json-parser","parser","security"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lovasoa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-10-21T22:15:17.000Z","updated_at":"2025-02-16T12:47:09.000Z","dependencies_parsed_at":"2022-07-27T22:48:14.147Z","dependency_job_id":null,"html_url":"https://github.com/lovasoa/bad_json_parsers","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lovasoa%2Fbad_json_parsers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lovasoa%2Fbad_json_parsers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lovasoa%2Fbad_json_parsers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lovasoa%2Fbad_json_parsers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lovasoa","download_url":"https://codeload.github.com/lovasoa/bad_json_parsers/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247406091,"owners_count":20933803,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","json-parser","parser","security"],"created_at":"2024-08-01T20:01:20.859Z","updated_at":"2025-04-05T22:08:36.179Z","avatar_url":"https://github.com/lovasoa.png","language":"Python","funding_links":[],"categories":["HarmonyOS","Python","Secure Programming"],"sub_categories":["Windows Manager","API"],"readme":"# Nesting levels for JSON parsers\n[![Build Status](https://travis-ci.org/lovasoa/bad_json_parsers.svg?branch=master)](https://travis-ci.org/lovasoa/bad_json_parsers)\n\nDocumenting how JSON parsers of several programming languages deal with deeply nested structures.\n\n## Introduction\n\nMany JSON parsers (and many parsers in general) use [recursion](https://en.wikipedia.org/wiki/Recursion_(computer_science))\nto parse nested structures.\nThis is very convenient while programming the parser, but it has consequences on what the parser can parse:\nindeed, the size of the [call stack](https://en.wikipedia.org/wiki/Call_stack) is usually limited to a value several orders of magnitude smaller\nthan the available RAM, and this implies that a program with too many levels of recursion will fail.\n\nThe two most recent JSON standards [RFC 8259](https://tools.ietf.org/html/rfc8259) and [RFC 7159](https://tools.ietf.org/html/rfc7159) both say \"An implementation may set limits on the maximum depth of nesting\". \nHowever, the [ECMA-404](http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf) specification\ndoesn't contain any limit on how deeply nested JSON structures can be. \n\nThis means that there is not a defined level of nesting which is correct or incorrect with regard to the JSON specification, and JSON parsers may differ when parsing nested structures.\n\nSome recursive parser libraries implement a safety check in order to avoid crashing the calling program:\nthey artificially limit the maximum depth they accept (often making that limit configurable),\nhoping that the size of the stack at the moment they are called plus the artificial limit will always be smaller than the total stack size.\nThis limit is an arbitrary choice of the library implementer, and it explains all the lower values of the comparison you'll see below.\n\nSome parsers do not use the operating system stack at all to parse nested structures\n(they usually implement a [state machine](https://en.wikipedia.org/wiki/Finite-state_machine) instead).\nThese can usually accept arbitrarily deeply nested structures.\nOf course, for non-streaming parsers, they cannot physically be provided infinitely large inputs,\nand thus cannot produce infinitely-large outputs.\n\nYou should note that parsers that set an arbitrary limit on the input nesting level are not safer\nand do not provide any more memory consumption guarantees than parsers that can handle arbitrarily nested input:\nthey still consume an amount of resources proportional to the size of their input. \n\nThis repository contains tools to measure the nesting limits of JSON parsers of different languages.\n\n## How to use\n\nThis repository contains a script called [test_parser.py](test_parser.py)\nthat takes a JSON parser and uses [binary search](https://en.wikipedia.org/wiki/Binary_search_algorithm)\nto find the smallest JSON structure it fails to parse and print its nesting level.\n\nThe json parser must be a program that reads JSON on its standard input,\nexits with a status of 0 if it managed to parse it and any other status if an error occurred.\n\n## How it works\n\n[test_parser.py](test_parser.py) constructs json structures composed uniquely of nested arrays,\nand gives them to the program it tests.\nFor instance, for a depth of 3, it builds the following json : `[[[]]]`.\nThis allows to create a structure of only *2n* bytes that has *n* nesting levels.\nIt uses [binary search](https://en.wikipedia.org/wiki/Binary_search_algorithm)\nto find the smallest structure for which the program fails.\n\n## Results\n\nThe various implementations in this repository are continuously\n[tested by Travis CI](https://travis-ci.org/lovasoa/bad_json_parsers)\non a virtual machine running Ubuntu 18.04, with 8Gb of RAM,\nand a maximum stack size of 8.192 Mb.\n\nHere are the results we found, sorted from least nesting allowed by default to the most:\n\nlanguage        | json library                                                | nesting level | file size     | notes                         |\n----------------| ----------------------------------------------------------- | ------------- | ------------- | ----\nC#              | [System.Text.Json](https://docs.microsoft.com/en-us/dotnet/api/system.text.json) | 65 | 130 bytes | configurable (`JsonSerializerOptions.MaxDepth`) *\nruby            | [json](https://rubygems.org/gems/json/versions/1.8.3)       | 101           | 202 bytes     | configurable (`:max_nesting`) *\nrust            | [serde\\_json](https://docs.serde.rs/serde_json/)            | 128           | 256 bytes     | disableable (`disable_recursion_limit`) *\nshell           | [jq](https://stedolan.github.io/jq/)                        | 257           | 514 bytes     | undocumented\nphp             | `json_decode`                                               | 512           | 1.0 KB    | configurable (`$depth`) *\nperl            | [JSON::PP](https://perldoc.perl.org/JSON/PP.html)           | 513           | 1.0 KB    | configurable (`max_depth`) *\nswift           | `JSONDecoder`                                               | 514           | 1.0 KB        | undocumented\npython3         | [json](https://docs.python.org/3/library/json.html)         | 995           | 2.0 KB        | configurable (`sys.setrecursionlimit`) \\*, undocumented\nC               | [jansson](https://jansson.readthedocs.io/)                  | 2049          | 4.0 KB        |\njavascript      | `JSON.parse`                                                | 5712          | 11.4 KB       | Node.js 8 LTS\njava            | [Gson](https://github.com/google/gson)                      | 6100          | 12   KB       |\njava            | [Jackson](https://github.com/FasterXML/jackson-core)        | 6577          | 13   KB       |\ngo              | [json-iterator](https://github.com/json-iterator/go)        | 10002         | 20   KB       | configurable (`Config.MaxDepth`) \\*\nPostgreSQL      | [json type](//postgresql.org/docs/10/datatype-json.html)    | 11887         | 23   KB       | configurable (`max_stack_depth`), undocumented\nD               | [`std.json`](https://dlang.org/phobos/std_json.html)        | 37370         | 74.7 KB       | segfaults\nC++             | [RapidJSON](http://rapidjson.org/)                          | 87266         | 175 KB        | segfaults\nNim             | [json](https://nim-lang.org/docs/json.html)                 | 104769        | 209 KB        | segfaults\nOCaml           | [yojson](https://github.com/ocaml-community/yojson)         | 130380        | 260 KB        |\ngo              | `encoding/json`                                             | 1973784       | 3.9 MiB       | fatal error, goroutine stack exceeds 1000000000-byte limit\nC++             | [JSON for Modern C++](https://github.com/nlohmann/json)     | ∞             | ∞             | segfault fixed in v3.7.2\nC#              | [Newtonsoft.Json](https://www.newtonsoft.com/json)          | ∞             | ∞             |\nruby            | [Oj](https://github.com/ohler55/oj)                         | ∞             | ∞             |\nHaskell         | [Aeson](https://hackage.haskell.org/package/aeson)          | ∞             | ∞             |\n\n\\* Note that *configurable* and *disableable* mean only that the default depth check inside the parser itself can be configured or disabled, not that the parser can be made to accept any nesting depth. When disabling the limit or increasing it too much, the parser will crash the calling program instead of returning a clean error. \n\n## Remarks\n\nI tried to test the most popular json library of each language. If you want to add a new language or a new library,\nfeel free to open a pull request. All the parameters were left to their default values.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flovasoa%2Fbad_json_parsers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flovasoa%2Fbad_json_parsers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flovasoa%2Fbad_json_parsers/lists"}