{"id":13611877,"url":"https://github.com/travisstaloch/simdjzon","last_synced_at":"2025-04-06T00:08:26.380Z","repository":{"id":44639276,"uuid":"393630279","full_name":"travisstaloch/simdjzon","owner":"travisstaloch","description":"simdjson ported to zig","archived":false,"fork":false,"pushed_at":"2025-03-05T21:42:25.000Z","size":755,"stargazers_count":118,"open_issues_count":5,"forks_count":6,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-29T23:08:39.050Z","etag":null,"topics":["zig-lang","zig-library","zig-package"],"latest_commit_sha":null,"homepage":"","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/travisstaloch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-07T08:50:09.000Z","updated_at":"2025-03-22T18:19:08.000Z","dependencies_parsed_at":"2023-01-31T17:00:50.084Z","dependency_job_id":"83636e28-34c6-4b36-865b-63f70a9d5faf","html_url":"https://github.com/travisstaloch/simdjzon","commit_stats":{"total_commits":149,"total_committers":4,"mean_commits":37.25,"dds":0.261744966442953,"last_synced_commit":"22d85a1699fa95346358cd1811cabf6adcd65279"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisstaloch%2Fsimdjzon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisstaloch%2Fsimdjzon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisstaloch%2Fsimdjzon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisstaloch%2Fsimdjzon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/travisstaloch","download_url":"https://codeload.github.com/travisstaloch/simdjzon/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247415967,"owners_count":20935387,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["zig-lang","zig-library","zig-package"],"created_at":"2024-08-01T19:02:16.916Z","updated_at":"2025-04-06T00:08:26.358Z","avatar_url":"https://github.com/travisstaloch.png","language":"Zig","funding_links":[],"categories":["Zig"],"sub_categories":[],"readme":":warning: **Work in progress.  Expect bugs and/or missing features** :warning:\n\n# simdjzon\nThis is a port of [simdjson](https://github.com/simdjson/simdjson), a high performance JSON parser developed by Daniel Lemire and Geoff Langdale to [zig](https://ziglang.org/).  \n\n# cpu support\nOnly 64 bit CPUs are supported so far.\n\n#### x86_64\nA CPU with AVX is required and CLMUL is preferred. the following usually have\nboth\n* Intel - Haswell from 2013 onwards\n* AMD - Ryzen/EPYC CPU (Q1 2017)\n\nThese commands show how to test for specific target and cpu support\n```console\nzig build test -Dtarget=x86_64-linux -Dcpu=x86_64+avx # uses clmulSoft - missing pclmul\nzig build test -Dtarget=x86_64-linux -Dcpu=x86_64+avx+pclmul\n# zig build test -Dtarget=x86_64-linux # doesn't work - missing avx\n# zig build test -Dcpu=x86_64_v2 # doesn't work - missing avx\n```\n\n#### aarch64\nA CPU with AES is preferred.\n```console\nzig build test -Dtarget=aarch64-linux -Dcpu=apple_latest-aes -fqemu # uses clmulSoft\nzig build test -Dtarget=aarch64-linux -fqemu\n```\n\n#### powerpc\nNot supported yet\n```console\n# zig build test -Dtarget=powerpc-linux -fqemu # doesn't work - no classify() + 32bit errors\n# zig build test -Dtarget=powerpc64-linux -fqemu # doesn't work - no classify()\n```\n\n#### fallback\nNo fallback for unsupported CPUs is provided yet.\n\n```console\n# zig build test -Dcpu=baseline # doesn't work - no classify()\n```\n\n# zig compiler support\nThe main branch is meant to compile with zig's master branch.  It is tested weekly on linux, windows and macos. \n\nFor older compiler versions, use a [tagged version](https://github.com/travisstaloch/simdjzon/tags).\n\n# usage\n### Git Clone\n```console\n# json validation\n$ git clone https://github.com/travisstaloch/simdjzon\n$ cd simdjzon\n$ zig build -Drelease-fast # uses the dom api by default\n$ zig-out/bin/simdjzon test/test.json\n$ echo $? # 0 on success\n0\n$ zig build -Drelease-fast -Dondemand # use the ondemand api\n$ zig-out/bin/simdjzon test/test.json\n$ echo $? # 0 on success\n0\n$ zig build test\nAll 19 tests passed.\n```\n\n### Zig Package\n```console\n$ $ zig fetch --save git+https://github.com/travisstaloch/simdjzon\ninfo: resolved to commit 28d46c979f761b211539232378138b692ef50d55\n```\n```zig\n// build.zig\nconst simdjzon_dep = b.dependency(\"simdjzon\", .{ .target = target, .optimize = optimize });\nconst exe_mod = b.createModule(.{\n    // ...\n    .imports = \u0026.{\n        .{ .name = \"simdjzon\", .module = simdjzon_dep.module(\"simdjzon\") },\n    },\n});\n```\n```zig\n// main.zig\nconst simdjzon = @import(\"simdjzon\");\n```\n\n```zig\nconst dom = @import(\"dom.zig\");\ntest \"get with struct\" {\n    const S = struct { a: u8, b: []const u8, c: struct { d: u8 } };\n    const input =\n        \\\\{\"a\": 42, \"b\": \"b-string\", \"c\": {\"d\": 126}}\n    ;\n    var parser = try dom.Parser.initFixedBuffer(allr, input, .{});\n    defer parser.deinit();\n    try parser.parse();\n    var s: S = undefined;\n    try parser.element().get(\u0026s);\n    try testing.expectEqual(@as(u8, 42), s.a);\n    try testing.expectEqualStrings(\"b-string\", s.b);\n    try testing.expectEqual(@as(u8, 126), s.c.d);\n}\n\ntest \"at_pointer\" {\n    const input =\n        \\\\{\"a\": {\"b\": [1,2,3]}}\n    ;\n    var parser = try dom.Parser.initFixedBuffer(allr, input, .{});\n    defer parser.deinit();\n    try parser.parse();\n    const b0 = try parser.element().at_pointer(\"/a/b/0\");\n    try testing.expectEqual(@as(i64, 1), try b0.get_int64());\n}\n\nconst ondemand = @import(\"ondemand.zig\");\n// ondemand api users must specify `pub const read_buf_cap = N;` in their\n// root source file.  In tests, this defaults to `std.mem.page_size`. \ntest \"ondemand get with struct\" {\n    const S = struct { a: struct { b: []const u8 } };\n    const input =\n        \\\\{\"a\": {\"b\": \"b-string\"}}\n    ;\n    var src = std.io.StreamSource{ .const_buffer = std.io.fixedBufferStream(input) };\n    var parser = try ondemand.Parser.init(\u0026src, allr, \"\u003cfba\u003e\", .{});\n    defer parser.deinit();\n    var doc = try parser.iterate();\n\n    var s: S = undefined;\n    try doc.get(\u0026s, .{ .allocator = allr });\n    defer allr.free(s.a.b);\n    try testing.expectEqualStrings(\"b-string\", s.a.b);\n}\n\ntest \"ondemand at_pointer\" {\n    const input =\n        \\\\{\"a\": {\"b\": [1,2,3]}}\n    ;\n    var src = std.io.StreamSource{ .const_buffer = std.io.fixedBufferStream(input) };\n    var parser = try ondemand.Parser.init(\u0026src, allr, \"\u003cfba\u003e\", .{});\n    defer parser.deinit();\n    var doc = try parser.iterate();\n    var b0 = try doc.at_pointer(\"/a/b/0\");\n    try testing.expectEqual(@as(u8, 1), try b0.get_int(u8));\n}\n```\n\n# performance\n## parsing/validating twitter.json (630Kb)\n### simdjson\n\n```console\n$ wget https://raw.githubusercontent.com/simdjson/simdjson/master/singleheader/simdjson.h https://raw.githubusercontent.com/simdjson/simdjson/master/singleheader/simdjson.cpp https://raw.githubusercontent.com/simdjson/simdjson/master/jsonexamples/twitter.json\n\n$ cat main.cpp\n#include \"simdjson.h\"\nusing namespace simdjson;\nint main(int argc, char** argv) {\n    if(argc != 2) {\n        std::cout \u003c\u003c \"USAGE: ./simdjson \u003cfile.json\u003e\" \u003c\u003c std::endl;\n        exit(1);\n    }\n    dom::parser parser; \n    try\n    {\n        const dom::element doc = parser.load(argv[1]);\n    }\n    catch(const std::exception\u0026 e)\n    {\n        std::cerr \u003c\u003c e.what() \u003c\u003c '\\n';\n        return 1;\n    }\n    return 0;\n}\n\n$ g++ main.cpp simdjson.cpp -o simdjson -O3 -march=native\n$ time ./simdjson twitter.json\n\nreal\t0m0.003s\nuser\t0m0.002s\nsys\t0m0.001s\n\n$ echo $?\n0\n```\n\n### simdjzon\n```console\n$ time zig-out/bin/simdjzon twitter.json \n\nreal\t0m0.002s\nuser\t0m0.000s\nsys\t0m0.002s\n\n$ echo $?\n0\n\n```\n\n### timed against simdjson, go, nim, zig std lib\nThe simdjson binary was compiled as shown above.  Go and nim binaries created with sources from JSONTestSuite. [zig std lib driver](bench/src/zig_json.zig).\nValidation times for several large json files.  Created with [benchmark_and_plot.jl](bench/benchmark_and_plot.jl)\n\n![results](https://github.com/travisstaloch/simdjson-zig/blob/media/bench/validation_grouped.png)\n\n# JSONTestSuite\n\nResults of running simdjson and simdjzon through [JSONTestSuite](https://github.com/nst/JSONTestSuite).  Results are equal as of 8/7/21\n\n![results](https://github.com/travisstaloch/simdjson-zig/blob/media/JSONTestSuiteResults.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftravisstaloch%2Fsimdjzon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftravisstaloch%2Fsimdjzon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftravisstaloch%2Fsimdjzon/lists"}