{"id":20799835,"url":"https://github.com/xnacly/libjson","last_synced_at":"2026-02-09T15:01:45.825Z","repository":{"id":252454544,"uuid":"840475441","full_name":"xnacly/libjson","owner":"xnacly","description":"High performance Go JSON parsing library","archived":false,"fork":false,"pushed_at":"2025-12-26T23:57:20.000Z","size":58,"stargazers_count":4,"open_issues_count":2,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-28T10:47:57.003Z","etag":null,"topics":["go","golang","json"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xnacly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-08-09T19:47:15.000Z","updated_at":"2025-12-26T19:13:26.000Z","dependencies_parsed_at":"2025-01-10T22:40:01.873Z","dependency_job_id":"ab1465d7-dadd-4f41-95b0-6aa5b4525dc2","html_url":"https://github.com/xnacly/libjson","commit_stats":null,"previous_names":["xnacly/gojson","xnacly/libjson"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/xnacly/libjson","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xnacly%2Flibjson","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xnacly%2Flibjson/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xnacly%2Flibjson/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xnacly%2Flibjson/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xnacly","download_url":"https://codeload.github.com/xnacly/libjson/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xnacly%2Flibjson/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29270147,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-09T13:47:44.167Z","status":"ssl_error","status_checked_at":"2026-02-09T13:47:43.721Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","json"],"created_at":"2024-11-17T18:10:22.027Z","updated_at":"2026-02-09T15:01:45.817Z","avatar_url":"https://github.com/xnacly.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# libjson\n\n\u003e WARNING: libjson is currently a work in progress :)\n\nFast and minimal JSON parser written in and for Go with a JIT query language\n\n```go\npackage main\n\nimport (\n    \"github.com/xnacly/libjson\"\n)\n\nfunc main() {\n\tinput := `{ \"hello\": {\"world\": [\"hi\"] } }`\n\tjsonObj, _ := New(input) // or libjson.NewReader(r io.Reader)\n\n\t// accessing values\n\tfmt.Println(Get[string](jsonObj, \".hello.world.0\")) // hi, nil\n}\n```\n\n## Features\n\n- [ECMA 404](https://ecma-international.org/wp-content/uploads/ECMA-404_2nd_edition_december_2017.pdf)\n  and [rfc8259](https://www.rfc-editor.org/rfc/rfc8259) compliant\n  - tests against [JSONTestSuite](https://github.com/nst/JSONTestSuite), see\n    [Parsing JSON is a Minefield\n    💣](https://seriot.ch/projects/parsing_json.html)in the future\n  - no trailing commata, comments, `Nan` or `Infinity`\n  - top level atom/skalars, like strings, numbers, true, false and null\n  - uft8 support via go [rune](https://go.dev/blog/strings)\n- no reflection, uses a custom query language similar to JavaScript object access instead\n- generics for value insertion and extraction with `libjson.Get` and `libjson.Set`\n- caching of queries with `libjson.Compile`, just in time caching of queries\n- serialisation via `json.Marshal`\n\n## Benchmarks\n\n![libjson-vs-encodingjson](https://github.com/user-attachments/assets/b11bcce4-e7db-4c45-ab42-45a2042e2a51)\n\n\nThese results were generated with the following specs:\n\n```text\nOS: Arch Linux x86_64\nKernel: 6.10.4-arch2-1\nMemory: 32024MiB\nGo version: 1.23\n```\n\nBelow this section is a list of performance improvements and their impact on\nthe overall performance as well as the full results of\n[test/bench.sh](test/bench.sh).\n\n### [b23001e](https://github.com/xNaCly/libjson/commit/b23001eca470935976a36cfbbc7a3c773d784a03)\n\n| JSON size | `encoding/json` | `libjson` |\n| --------- | --------------- | --------- |\n| 1MB       | 24.2ms          | 11.5ms    |\n| 5MB       | 117.3ms         | 48.5ms    |\n| 10MB      | 225ms           | 91ms      |\n\n- manually inlined `parser::expect`\n\n### [0058abb](https://github.com/xNaCly/libjson/commit/0058abb7381735b27783f9809947d7e0f22d9b05)\n\n| JSON size | `encoding/json` | `libjson` |\n| --------- | --------------- | --------- |\n| 1MB       | 24.2ms          | 12.0ms    |\n| 5MB       | 117.3ms         | 49.8ms    |\n| 10MB      | 225ms           | 93.8ms    |\n\n- replaced byte slices with offsets and lengths in the `token` struct\n\n### [88c5eb9](https://github.com/xNaCly/libjson/commit/88c5eb91c4fb1586af29b2cab3563b6ade424323)\n\n| JSON size | `encoding/json` | `libjson` |\n| --------- | --------------- | --------- |\n| 1MB       | 25.2ms          | 12.0ms    |\n| 5MB       | 117.3ms         | 50ms      |\n| 10MB      | 227ms           | 96ms      |\n\nThis commit made the tests more comparably by actually unmarshalling json into\na go data structure.\n\n### [a36a1bd](https://github.com/xNaCly/libjson/commit/a36a1bd042b10ce779c95c7c1e52232cf8d16fab)\n\n| JSON size | `encoding/json` | `libjson` |\n| --------- | --------------- | --------- |\n| 1MB       | 12.0ms          | 13.4ms    |\n| 5MB       | 58.4ms          | 66.3ms    |\n| 10MB      | 114.0ms         | 127.0ms   |\n\n- switch `token.Val` from `string` to `[]byte`, allows zero values to be `nil` and not `\"\"`\n- move string allocation for `t_string` and `t_number` to `(*parser).atom()`\n\n### [58e19ff](https://github.com/xNaCly/libjson/commit/58e19ffa140b01ff873505cb500364c4fea566db)\n\n| JSON size | `encoding/json` | `libjson` |\n| --------- | --------------- | --------- |\n| 1MB       | 12.3ms          | 14.2ms    |\n| 5MB       | 59.6ms          | 68.8ms    |\n| 10MB      | 115.3ms         | 131.8ms   |\n\nThe changes below resulted in the following savings: \\~6ms for 1MB, \\~25ms for\n5MB and \\~60ms for 10MB.\n\n- reuse buffer `lexer.buf` for number and string processing\n- switch from `(*bufio.Reader).ReadRune()` to `(*bufio.Reader).ReadByte()`\n- used `*(*string)(unsafe.Pointer(\u0026l.buf))` to skip strings.Builder usage for\n  number and string processing\n- remove and inline buffer usage for null, true and false, skipping allocations\n- benchmark the optimal initial cap for `lexer.buf`, maps and arrays to be 8\n- remove `errors.Is` and check for `t_eof` instead\n- move number parsing to `(*parser).atom()` and change type of `token.Val` to string,\n  this saves a lot of assertions, etc\n\n### [58d9360](https://github.com/xNaCly/libjson/commit/58d9360bae0576e761e021ee52035713206fdab1)\n\n| JSON size | `encoding/json` | `libjson` |\n| --------- | --------------- | --------- |\n| 1MB       | 12.2ms          | 19.9ms    |\n| 5MB       | 60.2ms          | 95.2ms    |\n| 10MB      | 117.2ms         | 183.8ms   |\n\nI had to change some things to account for issues occuring in the reading of\natoms, such as true, false and null. All of those are read by buffering the\nsize of chars they have and reading this buffer at once, instead of iterating\nand multiple reads. This did not work correctly because i used\n`(*bufio.Reader).Read`, which sometimes does not read all bytes fitting in the\nbuffer passed into it. Thats why these commit introduces a lot of performance\nregressions.\n\n### [e08beba](https://github.com/xNaCly/libjson/commit/e08bebada39441d9b6a20cb05251488ddce68285)\n\n| JSON size | `encoding/json` | `libjson` |\n| --------- | --------------- | --------- |\n| 1MB       | 11.7ms          | 13.1ms    |\n| 5MB       | 55.2ms          | 64.8ms    |\n\nThe optimisation in this commit is to no longer tokenize the whole input before\nstarting the parser but attaching the lexer to the parser. This allows the\nparser to invoke the tokenization of the next token on demand, for instance\nonce the parser needs to advance. This reduces the runtime around 4ms for the\n1MB input and 14ms for 5MB, resulting in a 1.33x and a 1.22x runtime reduction,\npretty good for such a simple change.\n\n### [be686d2](https://github.com/xNaCly/libjson/commit/be686d2c85c07cdfa91295052db54001d8cd5cc8)\n\n| JSON size | `encoding/json` | `libjson` |\n| --------- | --------------- | --------- |\n| 1MB       | 11.7ms          | 17.4ms    |\n| 5MB       | 55.2ms          | 78.5ms    |\n\nFor the first naiive implementation, these results are fairly good and not too\nfar behind the `encoding/go` implementation, however there are some potential\nlow hanging fruit for performance improvements and I will invest some time into\nthem.\n\nNo specific optimisations made here, except removing the check for duplicate\nobject keys, because\n[rfc8259](https://www.rfc-editor.org/rfc/rfc8259) says:\n\n\u003e When the names within an object are not\n\u003e unique, the behavior of software that receives such an object is\n\u003e unpredictable. Many implementations report the last name/value pair only.\n\u003e Other implementations report an error or fail to parse the object, and some\n\u003e implementations report all of the name/value pairs, including duplicates.\n\nThus I can decide wheter or not I want to error on duplicate keys, or simply\nlet each duplicate key overwrite the previous value in the object, however\nchecking if a given key is already in the map/object requires that key to be\nhashed and the map to be indexed with that key, omitting this check saves us\nthese operations, thus making the parser faster for large objects.\n\n### Reproduce locally\n\n\u003e Make sure you have the go toolchain and python3 installed for this.\n\n```shell\ncd test/\nchmod +x ./bench.sh\n./bench.sh\n```\n\nOutput looks something like:\n\n```text\nfetching example data\nbuilding executable\nBenchmark 1: ./test ./1MB.json\n  Time (mean ± σ):      13.1 ms ±   0.2 ms    [User: 12.1 ms, System: 2.8 ms]\n  Range (min … max):    12.7 ms …  13.8 ms    210 runs\n\nBenchmark 2: ./test -libjson=false ./1MB.json\n  Time (mean ± σ):      11.7 ms ±   0.3 ms    [User: 9.5 ms, System: 2.1 ms]\n  Range (min … max):    11.1 ms …  12.7 ms    237 runs\n\nSummary\n  ./test -libjson=false ./1MB.json ran\n    1.12 ± 0.03 times faster than ./test ./1MB.json\nBenchmark 1: ./test ./5MB.json\n  Time (mean ± σ):      64.2 ms ±   0.9 ms    [User: 79.3 ms, System: 13.1 ms]\n  Range (min … max):    62.6 ms …  67.0 ms    46 runs\n\nBenchmark 2: ./test -libjson=false ./5MB.json\n  Time (mean ± σ):      55.2 ms ±   1.1 ms    [User: 51.3 ms, System: 6.3 ms]\n  Range (min … max):    53.6 ms …  58.0 ms    53 runs\n\nSummary\n  ./test -libjson=false ./5MB.json ran\n    1.16 ± 0.03 times faster than ./test ./5MB.json\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxnacly%2Flibjson","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxnacly%2Flibjson","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxnacly%2Flibjson/lists"}