{"id":19991467,"url":"https://github.com/EgorBo/SimdJsonSharp","last_synced_at":"2025-05-04T10:31:48.943Z","repository":{"id":40462144,"uuid":"172245971","full_name":"EgorBo/SimdJsonSharp","owner":"EgorBo","description":"C# bindings for lemire/simdjson (and full C# port)","archived":false,"fork":false,"pushed_at":"2022-12-08T05:26:44.000Z","size":4208,"stargazers_count":662,"open_issues_count":10,"forks_count":43,"subscribers_count":42,"default_branch":"master","last_synced_at":"2025-04-08T00:37:46.773Z","etag":null,"topics":["avx2","json","netcore3","simd"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EgorBo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-02-23T17:59:15.000Z","updated_at":"2025-04-01T11:20:43.000Z","dependencies_parsed_at":"2023-01-24T14:30:16.374Z","dependency_job_id":null,"html_url":"https://github.com/EgorBo/SimdJsonSharp","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EgorBo%2FSimdJsonSharp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EgorBo%2FSimdJsonSharp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EgorBo%2FSimdJsonSharp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EgorBo%2FSimdJsonSharp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EgorBo","download_url":"https://codeload.github.com/EgorBo/SimdJsonSharp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252320368,"owners_count":21729114,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["avx2","json","netcore3","simd"],"created_at":"2024-11-13T04:51:46.664Z","updated_at":"2025-05-04T10:31:43.929Z","avatar_url":"https://github.com/EgorBo.png","language":"C#","funding_links":[],"categories":["C\\#"],"sub_categories":[],"readme":"# SimdJsonSharp: Parsing gigabytes of JSON per second\nC# version of [lemire/simdjson](https://github.com/lemire/simdjson) (by Daniel Lemire and Geoff Langdale - https://arxiv.org/abs/1902.08318) fully ported from C to C#, \nI tried to keep the same format and API). The library accelerates JSON parsing and minification using \nSIMD instructions (AVX2). C# version uses `System.Runtime.Intrinsics` API.\n\n**UPD:** Now it's also available as a set of pinvokes on top of the native lib as a .NETStandard 2.0 library, \nthus there are two implementations:\n1) [1.5.0](https://www.nuget.org/packages/SimdJsonSharp.Managed) Fully managed netcoreapp3.0 library (100% port from C to C#)\n2) [1.7.0](https://www.nuget.org/packages/SimdJsonSharp.Bindings) netstandard2.0 library with native lib (bindings are generated via [xoofx/CppAst](https://github.com/xoofx/CppAst))\n\n## Benchmarks\nThe following [benchmark](https://github.com/EgorBo/SimdJsonSharp/blob/master/benchmarks/CountTokens.cs) compares `SimdJsonSharp` with .NET Core 3.0 `Utf8JsonReader`, `Json.NET` and `SpanJson` libraries.\nTest json files can be found [here](https://github.com/lemire/simdjson/tree/master/jsonexamples).\n\n### 1. Parse doubles\nOpen [canada.json](https://raw.githubusercontent.com/lemire/simdjson/master/jsonexamples/canada.json) and parse all coordinates as `System.Double`:\n```\n|          Method |     fileName |    fileSize |      Mean | Ratio |\n|---------------- |------------- |-------------|----------:|------:|\n|        SimdJson |  canada.json | 2,251.05 Kb |  4,733 ms |  1.00 |\n|  Utf8JsonReader |  canada.json | 2,251.05 Kb | 56,692 ms | 11.98 |\n|         JsonNet |  canada.json | 2,251.05 Kb | 70,078 ms | 14.81 |\n|    SpanJsonUtf8 |  canada.json | 2,251.05 Kb | 54,878 ms | 11.60 |\n```\n\n### 2. Count all tokens\n```\n|            Method |           fileName |    fileSize |         Mean | Ratio |\n|------------------ |------------------- |------------ |-------------:|------:|\n|          SimdJson | apache_builds.json |   127.28 Kb |     99.28 us |  1.00 |\n|    Utf8JsonReader | apache_builds.json |   127.28 Kb |    226.42 us |  2.28 |\n|           JsonNet | apache_builds.json |   127.28 Kb |    461.30 us |  4.64 |\n|      SpanJsonUtf8 | apache_builds.json |   127.28 Kb |    168.08 us |  1.69 |\n|                   |                    |             |              |       |\n|          SimdJson |        canada.json | 2,251.05 Kb |  4,494.44 us |  1.00 |\n|    Utf8JsonReader |        canada.json | 2,251.05 Kb |  6,308.01 us |  1.40 |\n|           JsonNet |        canada.json | 2,251.05 Kb | 67,718.12 us | 15.06 |\n|      SpanJsonUtf8 |        canada.json | 2,251.05 Kb |  6,679.82 us |  1.49 |\n|                   |                    |             |              |       |\n|          SimdJson |  citm_catalog.json | 1,727.20 Kb |  1,572.78 us |  1.00 |\n|    Utf8JsonReader |  citm_catalog.json | 1,727.20 Kb |  3,786.10 us |  2.41 |\n|           JsonNet |  citm_catalog.json | 1,727.20 Kb |  5,903.38 us |  3.75 |\n|      SpanJsonUtf8 |  citm_catalog.json | 1,727.20 Kb |  3,021.13 us |  1.92 |\n|                   |                    |             |              |       |\n|          SimdJson | github_events.json |    65.13 Kb |     46.01 us |  1.00 |\n|    Utf8JsonReader | github_events.json |    65.13 Kb |    113.80 us |  2.47 |\n|           JsonNet | github_events.json |    65.13 Kb |    214.01 us |  4.65 |\n|      SpanJsonUtf8 | github_events.json |    65.13 Kb |     89.09 us |  1.94 |\n|                   |                    |             |              |       |\n|          SimdJson |     gsoc-2018.json | 3,327.83 Kb |  2,209.42 us |  1.00 |\n|    Utf8JsonReader |     gsoc-2018.json | 3,327.83 Kb |  4,010.10 us |  1.82 |\n|           JsonNet |     gsoc-2018.json | 3,327.83 Kb |  6,729.44 us |  3.05 |\n|      SpanJsonUtf8 |     gsoc-2018.json | 3,327.83 Kb |  2,759.59 us |  1.25 |\n|                   |                    |             |              |       |\n|          SimdJson |   instruments.json |   220.35 Kb |    257.78 us |  1.00 |\n|    Utf8JsonReader |   instruments.json |   220.35 Kb |    594.22 us |  2.31 |\n|           JsonNet |   instruments.json |   220.35 Kb |    980.42 us |  3.80 |\n|      SpanJsonUtf8 |   instruments.json |   220.35 Kb |    409.47 us |  1.59 |\n|                   |                    |             |              |       |\n|          SimdJson |      truenull.json |    12.00 Kb |  16,032.6 ns |  1.00 |\n|    Utf8JsonReader |      truenull.json |    12.00 Kb |  58,365.2 ns |  3.64 |\n|           JsonNet |      truenull.json |    12.00 Kb |  60,977.3 ns |  3.80 |\n|      SpanJsonUtf8 |      truenull.json |    12.00 Kb |  24,069.2 ns |  1.50 |\n```\n### 3. Json minification:\n```\n|                Method |           fileName |    fileSize |         Mean | Ratio |\n|---------------------- |------------------- |------------ |-------------:|------:|\n|  SimdJsonNoValidation | apache_builds.json |   127.28 Kb |     186.8 us |  1.00 |\n|              SimdJson | apache_builds.json |   127.28 Kb |     262.5 us |  1.41 |\n|               JsonNet | apache_builds.json |   127.28 Kb |   1,802.6 us |  9.65 |\n|                       |                    |             |              |       |\n|  SimdJsonNoValidation |        canada.json | 2,251.05 Kb |   4,130.7 us |  1.00 |\n|              SimdJson |        canada.json | 2,251.05 Kb |   7,940.7 us |  1.92 |\n|               JsonNet |        canada.json | 2,251.05 Kb | 181,884.0 us | 44.06 |\n|                       |                    |             |              |       |\n|  SimdJsonNoValidation |  citm_catalog.json | 1,727.20 Kb |   2,346.9 us |  1.00 |\n|              SimdJson |  citm_catalog.json | 1,727.20 Kb |   4,064.0 us |  1.75 |\n|               JsonNet |  citm_catalog.json | 1,727.20 Kb |  34,831.0 us | 14.84 |\n```\n\n## Usage\nThe C# API is not stable yet and currently fully copies the original C-style API\nthus it involves some `Unsafe` magic including pointers.\n\nAdd nuget package [SimdJsonSharp.Managed](https://www.nuget.org/packages/SimdJsonSharp.Managed) (for .NET Core 3.0)\nor [SimdJsonSharp.Bindings](https://www.nuget.org/packages/SimdJsonSharp.Bindings) for a .NETStandard 2.0 package (.NET 4.x, .NET Core 2.x, etc).\n```\ndotnet add package SimdJsonSharp.Bindings\nor\ndotnet add package SimdJsonSharp.Managed\n```\n\nThe following sample parses a file and iterate numeric tokens\n```csharp\nbyte[] bytes = File.ReadAllBytes(somefile);\nfixed (byte* ptr = bytes) // pin bytes while we are working on them\nusing (ParsedJson doc = SimdJson.ParseJson(ptr, bytes.Length))\nusing (var iterator = doc.CreateIterator())\n{\n    while (iterator.MoveForward())\n    {\n        if (iterator.GetTokenType() == JsonTokenType.Number)\n            Console.WriteLine(\"integer: \" + iterator.GetInteger());\n    }\n}\n```\n**UPD:** for `SimdJsonSharp.Bindings` types are postfixed with 'N', e.g. `ParsedJsonN`\n\nAs you can see the API looks similiar to `Utf8JsonReader` that was introduced recently in .NET Core 3.0\n\nAlso it's possible to just validate JSON or minify it (remove whitespaces, etc):\n```csharp\nstring someJson = ...;\nstring minifiedJson = SimdJson.MinifyJson(someJson);\n```\n\n## Requirements\n* AVX2 enabled CPU \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEgorBo%2FSimdJsonSharp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FEgorBo%2FSimdJsonSharp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEgorBo%2FSimdJsonSharp/lists"}