{"id":47653506,"url":"https://github.com/creationix/rx","last_synced_at":"2026-04-17T09:01:37.554Z","repository":{"id":345405705,"uuid":"1185774518","full_name":"creationix/rx","owner":"creationix","description":"RX encoder, decoder, and CLI data tool","archived":false,"fork":false,"pushed_at":"2026-04-08T11:51:22.000Z","size":1101,"stargazers_count":376,"open_issues_count":1,"forks_count":6,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-04-08T13:27:32.603Z","etag":null,"topics":["database","random-access","rex","rx","serialization","text-encoding"],"latest_commit_sha":null,"homepage":"https://rx.run/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/creationix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-18T23:42:49.000Z","updated_at":"2026-04-08T11:51:26.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/creationix/rx","commit_stats":null,"previous_names":["creationix/rx"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/creationix/rx","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Frx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Frx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Frx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Frx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/creationix","download_url":"https://codeload.github.com/creationix/rx/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Frx/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31922399,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-16T18:22:33.417Z","status":"online","status_checked_at":"2026-04-17T02:00:06.879Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","random-access","rex","rx","serialization","text-encoding"],"created_at":"2026-04-02T08:00:43.401Z","updated_at":"2026-04-17T09:01:37.549Z","avatar_url":"https://github.com/creationix.png","language":"TypeScript","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"readme":"# RX Data Store\n\n[![rx tests](https://github.com/creationix/rx/actions/workflows/rx-test.yml/badge.svg)](https://github.com/creationix/rx/actions/workflows/rx-test.yml)\n\nRX is an embedded data store for JSON-shaped data. Encode once, then query the encoded document in place — **no parsing, no object graph, no GC pressure**. Think of it as *no-SQL SQLite*: unstructured data with database-style random access.\n\n```json\n[ { \"color\": \"red\", \"fruits\": [ \"apple\", \"cherry\" ] },\n  { \"color\": \"yellow\", \"fruits\": [ \"apple\", \"banana\" ] } ]\n```\n\nWhen encoding as RX, pointers deduplicate automatically: `^z` reuses \"apple\", `^h` reuses the shared key layout. The encoded form is queryable as-is — no parsing step, just direct reads from the buffer.\n\n```rexc\nbanana,6apple,5;ffruits,6yellow,6color,5:Echerry,6^z;ared,3^h:j;_\n```\n\nBenchmarked on a real **92 MB deployment manifest** with 35,000 route keys:\n\n| | JSON | RX |\n|---|------|-----|\n| **Size** | 92 MB | 5.1 MB |\n| **Look up one route** | 69 ms (full parse) | 0.003 ms (~16 index hops) |\n| **Heap allocations** | 2,598,384 | ~10 |\n\n---\n\n## When to use RX\n\nRX sits in a specific gap: your data is **too large** for JSON's parse-everything model, but **too unstructured** for SQLite or Protobuf.\n\n**Good fits:**\n- Build manifests, route tables, deployment artifacts — written once, read sparsely\n- Embedded datasets in browsers, edge runtimes, or worker processes\n- Any workflow where full-document parsing is the bottleneck\n\n**Bad fits:**\n- Small documents where JSON parsing is already cheap\n- Human-authored config files\n- Write-heavy or mutable data (use a real database)\n- Minimizing compressed transfer size (gzip/zstd will beat RX)\n- Data that maps cleanly to tables (use SQLite) or a fixed schema (use Protobuf)\n\n## Typical workflow\n\n1. A build or deploy step produces a large JSON-shaped artifact.\n2. **Encode it to RX** once.\n3. Runtimes read only the values they need — **O(1)** array access, **O(log n)** object key lookup.\n4. When debugging at 3 AM, copy-paste the RX text into **[rx.run](https://rx.run/)** to inspect it. No binary tooling needed.\n\n---\n\n## Install\n\n```sh\nnpm install @creationix/rx     # library\nnpm install -g @creationix/rx  # CLI (global)\nnpx @creationix/rx data.rx     # CLI (one-off)\n```\n\n## Quick start\n\n### Encode\n\n```ts\nimport { stringify } from \"@creationix/rx\";\n\nconst rx = stringify({ users: [\"alice\", \"bob\"], version: 3 });\n// Returns a string — store it anywhere you'd store JSON text\n```\n\n### Decode\n\n```ts\nimport { parse } from \"@creationix/rx\";\n\nconst data = parse(rx) as any;\ndata.users[0]         // \"alice\"  — no parse, direct buffer read\ndata.version          // 3\nObject.keys(data)     // [\"users\", \"version\"]\nJSON.stringify(data)  // works — full JS interop\n```\n\nThe returned value is a **read-only Proxy**. It supports property access, `Object.keys()`, `Object.entries()`, `for...of`, `for...in`, `Array.isArray()`, `.map()`, `.filter()`, `.find()`, `.reduce()`, spread, destructuring, and `JSON.stringify()`. Existing read paths usually work unchanged.\n\n### Uint8Array API\n\nFor performance-critical paths, skip the string conversion:\n\n```ts\nimport { encode, open } from \"@creationix/rx\";\n\nconst buf = encode({ path: \"/api/users\", status: 200 });\nconst data = open(buf) as any;\ndata.path    // \"/api/users\"\ndata.status  // 200\n```\n\n`stringify`/`parse` work with strings. `encode`/`open` work with `Uint8Array`. Same options, same Proxy behavior.\n\n### A more realistic example\n\nThe quick start above is tiny — JSON would be fine for it. RX pays off on **larger data with sparse reads**. Here's a site manifest (see [samples/](samples/) for full files):\n\n```js\n// site-manifest.json — 15 routes, repeated structure, shared prefixes\n{\n  \"routes\": {\n    \"/\": { \"title\": \"Home\", \"component\": \"LandingPage\", \"auth\": false },\n    \"/docs\": { \"title\": \"Documentation\", \"component\": \"DocsIndex\", \"auth\": false },\n    \"/docs/getting-started\": { \"title\": \"Getting Started\", \"component\": \"DocsPage\", \"auth\": false },\n    \"/dashboard\": { \"title\": \"Dashboard\", \"component\": \"Dashboard\", \"auth\": true },\n    \"/dashboard/projects\": { \"title\": \"Projects\", \"component\": \"ProjectList\", \"auth\": true },\n    // ... 10 more routes\n  }\n}\n```\n\n```ts\nimport { readFileSync } from \"fs\";\nimport { parse } from \"@creationix/rx\";\n\n// The RX file is already smaller on disk (shared schemas, deduplicated\n// component names, chain-compressed \"/docs/...\" and \"/dashboard/...\" prefixes).\n// But the real win is at read time:\n\nconst manifest = parse(readFileSync(\"site-manifest.rx\", \"utf-8\")) as any;\nconst route = manifest.routes[\"/dashboard/projects\"];\nroute.title      // \"Projects\"\nroute.component  // \"ProjectList\"\nroute.auth       // true\n// Only these three values were decoded. Everything else was skipped.\n```\n\nScale this to 35,000 routes and the difference is **69 ms vs 0.003 ms** per lookup.\n\nThe [samples/](samples/) directory has four datasets showing different access patterns — route manifests, RPG game state, emoji metadata, and sensor telemetry. Start with *site-manifest* and *quest-log* if you're evaluating the format.\n\n---\n\n## Encoding options\n\n```ts\nstringify(data, {\n  // Add sorted indexes to containers with \u003e= N entries (enables O(log n) lookup)\n  indexes: 10,       // default threshold; use 0 for all, false to disable\n\n  // External refs — shared dictionary of known values\n  refs: { R: [\"/api/users\", \"/api/teams\"] },\n\n  // Streaming — receive chunks as they're produced\n  onChunk: (chunk, offset) =\u003e process.stdout.write(chunk),\n});\n```\n\nIf the encoder used external refs, pass the same dictionary to the decoder:\n\n```ts\nconst data = parse(payload, { refs: { R: [\"/api/users\", \"/api/teams\"] } });\n```\n\n## CLI\n\n```sh\nrx data.rx                         # pretty-print as tree\nrx data.rx -j                      # convert to JSON\nrx data.json -r                    # convert to RX\ncat data.rx | rx                   # read from stdin (auto-detect)\nrx data.rx -s key 0 sub            # select a sub-value\nrx data.rx -o out.json             # write to file\nrx data.rx --ast                   # output encoding structure as JSON\n```\n\n\u003e **Tip:** Add a shell function for quick paged, colorized viewing:\n\u003e ```sh\n\u003e p() { rx \"$1\" -t -c | less -RFX; }\n\u003e ```\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eFull CLI reference\u003c/strong\u003e\u003c/summary\u003e\n\n| Flag | Description |\n|------|-------------|\n| `\u003cfile\u003e` | Input file (format auto-detected by contents) |\n| `-` | Read from stdin explicitly |\n| `-j`, `--json` | Output as JSON |\n| `-r`, `--rexc` | Output as RX |\n| `-t`, `--tree` | Output as tree (default on TTY) |\n| `-a`, `--ast` | Output encoding structure |\n| `-s`, `--select \u003cseg\u003e...` | Select a sub-value |\n| `-w`, `--write` | Write converted file (`.json`↔`.rx`) |\n| `-o`, `--out \u003cpath\u003e` | Write to file instead of stdout |\n| `-c`, `--color` / `--no-color` | Force or disable ANSI color |\n| `--index-threshold \u003cn\u003e` | Index containers above n values (default: 16) |\n| `--string-chain-threshold \u003cn\u003e` | Split strings longer than n into chains (default: 64) |\n| `--string-chain-delimiter \u003cs\u003e` | Delimiter for string chains (default: `/`) |\n| `--key-complexity-threshold \u003cn\u003e` | Max object complexity for dedupe keys (default: 100) |\n\nShell completions:\n\n```sh\nrx --completions setup [zsh|bash]\n```\n\n\u003c/details\u003e\n\n---\n\n## Format\n\nRX is a **text encoding** — not human-readable like JSON, but safe to copy-paste, embed in strings, and move through tools that choke on binary.\n\nEvery value is read **right-to-left**. The parser scans left past base64 digits to find a **tag** character, then uses the tag to interpret any **body** bytes further left:\n\n```\n[body][tag][b64 varint]\n            ◄── read this way ──\n```\n\n*Railroad diagram coming soon — see [format spec](docs/rx-format.md) for all diagrams.*\n\n| JSON | RX | What you're reading |\n|------|----|---------------------|\n| `42` | `+1k` | tag `+` (integer), b64 `1k` = 84, zigzag → 42 |\n| `\"hi\"` | `hi,2` | tag `,` (string), b64 `2` = byte length, body `hi` to the left |\n| `true` | `'t` | tag `'` (ref), name `t` → built-in literal |\n| `[1,2,3]` | `+6+4+2;6` | tag `;` (array), b64 `6` = content size, three children to the left |\n| `{\"a\":1,\"b\":2}` | `+4b,1+2a,1:a` | tag `:` (object), b64 `a` = content size, interleaved keys/values |\n\n**Tags:** `+` integer · `*` decimal · `,` string · `'` ref/literal · `:` object · `;` array · `^` pointer · `.` chain · `#` index\n\nThe encoder automatically deduplicates values, shares object schemas, compresses shared string prefixes, and adds sorted indexes. See the **[format spec](docs/rx-format.md)** for the full grammar, railroad diagrams, and a walkthrough of how a complete object is encoded byte by byte.\n\n![RX Viewer — interactive tree inspector](rexc-viewer-screenshot.png)\n\nTo inspect real data, paste RX or JSON into the live viewer at **[rx.run](https://rx.run/)**.\n\n---\n\n## Inspect API\n\n`inspect()` returns a lazy AST that maps 1:1 to the byte encoding — pointers stay as pointers, chains as chains, indexes as indexes:\n\n```ts\nimport { encode, inspect } from \"@creationix/rx\";\n\nconst buf = encode({ name: \"alice\", scores: [10, 20, 30] });\nconst root = inspect(buf);\n\nroot.tag          // \":\"\nroot[0].tag       // \",\" (a string key)\nroot[0].value     // \"name\"\nroot.length       // 4 (key, value, key, value)\n\nfor (const child of root) {\n  console.log(child.tag, child.b64);\n}\n```\n\nEach node exposes: `tag`, `b64`, `left`, `right`, `size`, `data`, and `value` (lazy). Nodes with children (`:`, `;`, `.`, `*`, `#`) are iterable and support indexed access. Children are parsed lazily and cached.\n\n**Semantic helpers** for object nodes:\n\n```ts\nfor (const [key, val] of root.entries()) { ... }\nfor (const [key, val] of root.filteredKeys(\"/api/\")) { ... }  // O(log n + m) on indexed objects\nconst node = root.index(\"name\");   // key lookup\nconst elem = root.index(2);        // array index\n```\n\n## Low-level cursor API\n\nFor zero-allocation traversal without the Proxy layer, see **[docs/cursor-api.md](docs/cursor-api.md)**.\n\n## Proxy behavior\n\nThe value returned by `parse`/`open` is **read-only**:\n\n```ts\nobj.newKey = 1;      // throws TypeError\ndelete obj.key;      // throws TypeError\n\"key\" in obj;        // works (zero-alloc key search)\nobj.nested === obj.nested  // true (container Proxies are memoized)\n```\n\nEscape hatch to the underlying buffer:\n\n```ts\nimport { handle } from \"@creationix/rx\";\nconst h = handle(obj.nested);\n// h.data: Uint8Array, h.right: byte offset\n```\n\n---\n\n## More\n\n- **[docs/rx-format.md](docs/rx-format.md)** — format spec, grammar, and railroad diagrams\n- **[docs/cursor-api.md](docs/cursor-api.md)** — low-level zero-allocation cursor API\n- **[rx-perf.md](rx-perf.md)** — cursor internals, Proxy design, allocation profile\n- **[samples/](samples/)** — example datasets with JSON/RX pairs\n- **[rx.run](https://rx.run/)** — live web viewer\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcreationix%2Frx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcreationix%2Frx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcreationix%2Frx/lists"}