{"id":47299076,"url":"https://github.com/lpgauth/torque","last_synced_at":"2026-05-14T21:04:40.614Z","repository":{"id":344900587,"uuid":"1183507064","full_name":"lpgauth/torque","owner":"lpgauth","description":"High-performance JSON library for Elixir via Rustler NIFs, powered by sonic-rs (SIMD-accelerated)","archived":false,"fork":false,"pushed_at":"2026-03-28T13:32:57.000Z","size":162,"stargazers_count":14,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-02T08:52:22.818Z","etag":null,"topics":["elixir","json","nif","sonic-rs"],"latest_commit_sha":null,"homepage":"https://hex.pm/packages/torque","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lpgauth.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-16T17:15:16.000Z","updated_at":"2026-03-31T02:30:34.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/lpgauth/torque","commit_stats":null,"previous_names":["lpgauth/torque"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/lpgauth/torque","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lpgauth%2Ftorque","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lpgauth%2Ftorque/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lpgauth%2Ftorque/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lpgauth%2Ftorque/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lpgauth","download_url":"https://codeload.github.com/lpgauth/torque/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lpgauth%2Ftorque/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33043249,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-13T13:14:54.681Z","status":"online","status_checked_at":"2026-05-14T02:00:06.663Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","json","nif","sonic-rs"],"created_at":"2026-03-16T22:00:31.975Z","updated_at":"2026-05-14T21:04:40.600Z","avatar_url":"https://github.com/lpgauth.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Torque\n\nHigh-performance JSON library for Elixir via [Rustler](https://github.com/rustler-magic/rustler) NIFs, powered by [sonic-rs](https://github.com/cloudwego/sonic-rs) (SIMD-accelerated).\n\nTorque provides the fastest JSON encoding and decoding available in the BEAM ecosystem, with a selective field extraction API for workloads that only need a subset of fields from each document.\n\n## Features\n\n- SIMD-accelerated decoding (AVX2/SSE4.2 on x86, NEON on ARM)\n- Ultra-low memory encoder (64 B per encode vs ~4 KB for OTP `json`/jason)\n- Parse-then-get API for selective field extraction via JSON Pointer (RFC 6901)\n- Batch field extraction (`get_many/2`) with single NIF call\n- Automatic dirty CPU scheduler dispatch for inputs larger than 20 KB\n- jiffy-compatible `{proplist}` encoding\n\n## Installation\n\nAdd to your `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:torque, \"~\u003e 0.1.9\"}\n  ]\nend\n```\n\nPrecompiled binaries are available for common targets. To compile from source, install a stable Rust toolchain and set `TORQUE_BUILD=true`.\n\n### CPU-optimized variants\n\nOn x86_64, precompiled binaries are available for three CPU feature levels:\n\n| Variant | CPU features | `target-cpu` |\n|---------|-------------|--------------|\n| baseline | SSE2 | `x86-64` |\n| v2 | SSE4.2, SSSE3, POPCNT | `x86-64-v2` |\n| v3 | AVX2, AVX, BMI1, BMI2, FMA | `x86-64-v3` |\n\nAt compile time, Torque auto-detects the host CPU and downloads the best matching variant. To override detection (e.g., when cross-compiling for a different target):\n\n```bash\nTORQUE_CPU_VARIANT=v2 mix compile  # force SSE4.2 variant\nTORQUE_CPU_VARIANT=v3 mix compile  # force AVX2 variant\nTORQUE_CPU_VARIANT=base mix compile  # force baseline\n```\n\n## Usage\n\n### Decoding\n\n```elixir\n{:ok, data} = Torque.decode(~s({\"name\":\"Alice\",\"age\":30}))\n# %{\"name\" =\u003e \"Alice\", \"age\" =\u003e 30}\n\ndata = Torque.decode!(json)\n```\n\n### Selective Field Extraction\n\nParse once, extract many fields without building the full Elixir term tree:\n\n```elixir\n{:ok, doc} = Torque.parse(json)\n\n{:ok, \"example.com\"} = Torque.get(doc, \"/site/domain\")\nnil = Torque.get(doc, \"/missing/field\", nil)\n\n# Batch extraction (single NIF call, fastest path)\nresults = Torque.get_many(doc, [\"/id\", \"/site/domain\", \"/device/ip\"])\n# [{:ok, \"req-1\"}, {:ok, \"example.com\"}, {:ok, \"1.2.3.4\"}]\n```\n\nWhen your JSON is known to have no duplicate object keys, pass `unique_keys: true`\nfor faster field lookups (uses sonic-rs internal indexing instead of linear scan):\n\n```elixir\n{:ok, doc} = Torque.parse(json, unique_keys: true)\n```\n\n### Encoding\n\n```elixir\n# Maps with atom or binary keys\n{:ok, json} = Torque.encode(%{id: \"abc\", price: 1.5})\n# \"{\\\"id\\\":\\\"abc\\\",\\\"price\\\":1.5}\"\n\n# Bang variant\njson = Torque.encode!(%{id: \"abc\"})\n\n# iodata variant (fastest, no {:ok, ...} tuple wrapping)\njson = Torque.encode_to_iodata(%{id: \"abc\"})\n\n# jiffy-compatible proplist format\n{:ok, json} = Torque.encode({[{:id, \"abc\"}, {:price, 1.5}]})\n```\n\n## API\n\n| Function | Description |\n|----------|-------------|\n| `Torque.decode(binary)` | Decode JSON to Elixir terms |\n| `Torque.decode!(binary)` | Decode JSON, raising on error |\n| `Torque.parse(binary, opts)` | Parse JSON into opaque document reference |\n| `Torque.get(doc, path)` | Extract field by JSON Pointer path |\n| `Torque.get(doc, path, default)` | Extract field with default for missing paths |\n| `Torque.get_many(doc, paths)` | Extract multiple fields in one NIF call |\n| `Torque.get_many_nil(doc, paths)` | Extract multiple fields, `nil` for missing |\n| `Torque.length(doc, path)` | Return length of array at path |\n| `Torque.encode(term)` | Encode term to JSON binary |\n| `Torque.encode!(term)` | Encode term, raising on error |\n| `Torque.encode_to_iodata(term)` | Encode term, returns binary directly (fastest) |\n\n## Type Conversion\n\n### JSON to Elixir\n\n| JSON | Elixir |\n|------|--------|\n| object | map (binary keys) |\n| array | list |\n| string | binary |\n| integer | integer |\n| float | float |\n| `true`, `false` | `true`, `false` |\n| `null` | `nil` |\n\nFor objects with duplicate keys, the last value wins (unless `unique_keys: true` is passed to `parse/2`).\n\n### Elixir to JSON\n\n| Elixir | JSON |\n|--------|------|\n| map (atom/binary keys) | object |\n| list | array |\n| binary | string |\n| integer | number |\n| float | number |\n| `true`, `false` | `true`, `false` |\n| `nil` | `null` |\n| atom | string |\n| `{keyword_list}` | object |\n\n## Errors\n\nFunctions return `{:error, reason}` tuples (or raise `ArgumentError` for bang/iodata variants). Possible `reason` atoms:\n\n### Decode / Parse\n\n| Atom | Returned by | Meaning |\n|------|-------------|---------|\n| `:nesting_too_deep` | `decode/1`, `get/2`, `get_many/2` | Document exceeds 512 nesting levels |\n\n`parse/1` and `decode/1` also return `{:error, binary}` with a message from sonic-rs for malformed JSON.\n\n### Encode\n\n| Atom | Returned by | Meaning |\n|------|-------------|---------|\n| `:unsupported_type` | `encode/1` | Term has no JSON representation (PID, reference, port, …) |\n| `:invalid_utf8` | `encode/1` | Binary string or map key is not valid UTF-8 |\n| `:invalid_key` | `encode/1` | Map key is not an atom or binary (e.g. integer key) |\n| `:malformed_proplist` | `encode/1` | `{proplist}` contains a non-`{key, value}` element |\n| `:non_finite_float` | `encode/1` | Float is infinity or NaN (unreachable from normal BEAM code) |\n| `:nesting_too_deep` | `encode/1` | Term exceeds 512 nesting levels |\n\n## Benchmarks\n\nApple M2 Pro, OTP 28, Elixir 1.19:\n\n### Decode (1.2 KB OpenRTB)\n\n| Library | ips | mean | median | p99 | memory |\n|---|---|---|---|---|---|\n| **torque** | **262.5K** | **3.81 μs** | **3.63 μs** | **7.83 μs** | **1.56 KB** |\n| **simdjsone** | 182.7K | 5.47 μs | 5.13 μs | 11.88 μs | 1.59 KB |\n| **jiffy** | 144.6K | 6.92 μs | 6.21 μs | 17.17 μs | **1.56 KB** |\n| **otp json** | 129.6K | 7.72 μs | 7.21 μs | 16.50 μs | 7.73 KB |\n| **jason** | 103.6K | 9.65 μs | 8.71 μs | 22.75 μs | 9.54 KB |\n\n### Decode (750 KB Twitter)\n\n| Library | ips | mean | median | p99 | memory |\n|---|---|---|---|---|---|\n| **torque** | **476.0** | **2.10 ms** | **1.87 ms** | **4.73 ms** | **1.56 KB** |\n| **simdjsone** | 459.4 | 2.18 ms | 1.85 ms | 3.20 ms | **1.56 KB** |\n| **otp json** | 195.1 | 5.13 ms | 5.12 ms | 6.16 ms | 2.49 MB |\n| **jason** | 142.0 | 7.04 ms | 6.91 ms | 11.47 ms | 3.55 MB |\n| **jiffy** | 115.9 | 8.63 ms | 8.72 ms | 9.94 ms | 5.53 MB |\n\n### Encode (1.2 KB OpenRTB)\n\n| Library | ips | mean | median | p99 | memory |\n|---|---|---|---|---|---|\n| **otp json** [map() :: iodata()] | **1091.6K** | **0.92 μs** | **0.83 μs** | 1.46 μs | 3928 B |\n| **torque** [proplist() :: binary()] | 1073.6K | 0.93 μs | 0.88 μs | **1.13 μs** | 88 B |\n| **torque** [proplist() :: iodata()] | 1069.3K | 0.94 μs | 0.88 μs | 1.17 μs | **64 B** |\n| **torque** [map() :: binary()] | 917.5K | 1.09 μs | 1.00 μs | 1.33 μs | 88 B |\n| **torque** [map() :: iodata()] | 914.6K | 1.09 μs | 1.00 μs | 1.42 μs | **64 B** |\n| **jason** [map() :: iodata()] | 571.8K | 1.75 μs | 1.54 μs | 3.75 μs | 3848 B |\n| **jiffy** [proplist() :: iodata()] | 518.4K | 1.93 μs | 1.67 μs | 2.75 μs | 120 B |\n| **jiffy** [map() :: iodata()] | 427.6K | 2.34 μs | 2.08 μs | 4.33 μs | 824 B |\n| **simdjsone** [proplist() :: iodata()] | 415.4K | 2.41 μs | 2.21 μs | 3.96 μs | 184 B |\n| **jason** [map() :: binary()] | 385.1K | 2.60 μs | 2.38 μs | 5.00 μs | 3912 B |\n| **simdjsone** [map() :: iodata()] | 346.8K | 2.88 μs | 2.67 μs | 4.33 μs | 888 B |\n\n### Encode (750 KB Twitter)\n\n| Library | ips | mean | median | p99 | memory |\n|---|---|---|---|---|---|\n| **torque** [proplist() :: iodata()] | **1026.4** | **0.97 ms** | **0.96 ms** | **1.18 ms** | **64 B** |\n| **torque** [proplist() :: binary()] | 983.5 | 1.02 ms | 0.98 ms | 1.69 ms | 88 B |\n| **torque** [map() :: binary()] | 918.5 | 1.09 ms | 1.08 ms | 1.31 ms | 88 B |\n| **torque** [map() :: iodata()] | 905.4 | 1.10 ms | 1.09 ms | 1.35 ms | **64 B** |\n| **jiffy** [proplist() :: iodata()] | 342.6 | 2.92 ms | 2.86 ms | 4.35 ms | 37.7 KB |\n| **jiffy** [map() :: iodata()] | 270.8 | 3.69 ms | 3.53 ms | 5.94 ms | 1.06 MB |\n| **jason** [map() :: iodata()] | 254.9 | 3.92 ms | 3.70 ms | 6.50 ms | 4.96 MB |\n| **simdjsone** [proplist() :: iodata()] | 247.4 | 4.04 ms | 3.98 ms | 5.63 ms | 37.7 KB |\n| **otp json** [map() :: iodata()] | 246.9 | 4.05 ms | 4.13 ms | 5.64 ms | 5.40 MB |\n| **simdjsone** [map() :: iodata()] | 210.5 | 4.75 ms | 4.78 ms | 5.41 ms | 1.06 MB |\n| **jason** [map() :: binary()] | 141.1 | 7.09 ms | 7.02 ms | 8.40 ms | 4.96 MB |\n\n### Parse (1.2 KB OpenRTB)\n\n| Library | ips | mean | median | p99 |\n|---|---|---|---|---|\n| **torque** parse(unique_keys) | **596.6K** | **1.68 μs** | **1.33 μs** | **3.13 μs** |\n| **torque** parse | 579.2K | 1.73 μs | **1.33 μs** | 3.88 μs |\n| **simdjsone** parse | 364.9K | 2.74 μs | 1.17 μs | 4.92 μs |\n\n### Get (5 fields) (1.2 KB OpenRTB)\n\n| Library | ips | mean | median | p99 | memory |\n|---|---|---|---|---|---|\n| **torque** get_many_nil (unique_keys) | **2.49M** | **402 ns** | **375 ns** | **500 ns** | **240 B** |\n| **torque** get_many (unique_keys) | 2.37M | 422 ns | **375 ns** | **500 ns** | 360 B |\n| **torque** get_many_nil | 2.16M | 463 ns | 458 ns | 583 ns | **240 B** |\n| **torque** get_many | 2.07M | 483 ns | 458 ns | 584 ns | 360 B |\n| **simdjsone** get | 1.77M | 564 ns | 458 ns | 1083 ns | 384 B |\n| **torque** get (unique_keys) | 1.67M | 601 ns | 583 ns | 709 ns | 384 B |\n| **torque** get | 1.50M | 669 ns | 625 ns | 792 ns | 384 B |\n\nRun benchmarks locally:\n\n```bash\nMIX_ENV=bench mix run bench/torque_bench.exs\n```\n\n## Limitations\n\n- **Nesting depth**: JSON documents nested deeper than 512 levels return `{:error, :nesting_too_deep}` from `decode/1`, `get/2`, `get_many/2`, and `encode/1` rather than crashing the VM. Real-world documents are never this deep; the limit exists to prevent stack overflow in the NIF.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flpgauth%2Ftorque","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flpgauth%2Ftorque","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flpgauth%2Ftorque/lists"}