{"id":22472206,"url":"https://github.com/ckampfe/jstream","last_synced_at":"2026-02-11T23:03:23.510Z","repository":{"id":235030334,"uuid":"788242147","full_name":"ckampfe/jstream","owner":"ckampfe","description":"Index JSON values by their paths in a document for easy grepping ","archived":false,"fork":false,"pushed_at":"2025-08-28T04:30:57.000Z","size":260,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-17T11:51:54.458Z","etag":null,"topics":["command-line-tool","grep","gron","index","json","search"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ckampfe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-04-18T03:25:37.000Z","updated_at":"2025-09-23T07:39:52.000Z","dependencies_parsed_at":"2024-04-22T03:16:30.150Z","dependency_job_id":"797008cb-481f-40a5-828a-844633b40c7a","html_url":"https://github.com/ckampfe/jstream","commit_stats":null,"previous_names":["ckampfe/jstream"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ckampfe/jstream","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckampfe%2Fjstream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckampfe%2Fjstream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckampfe%2Fjstream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckampfe%2Fjstream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ckampfe","download_url":"https://codeload.github.com/ckampfe/jstream/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ckampfe%2Fjstream/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29349259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-11T20:11:40.865Z","status":"ssl_error","status_checked_at":"2026-02-11T20:10:41.637Z","response_time":97,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["command-line-tool","grep","gron","index","json","search"],"created_at":"2024-12-06T12:12:39.646Z","updated_at":"2026-02-11T23:03:23.481Z","avatar_url":"https://github.com/ckampfe.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# jstream\n\nEnumerate the paths through a JSON document.\n\nThis project is very much like [gron](https://github.com/tomnomnom/gron) or my other project, [jindex](https://github.com/ckampfe/jindex), but this project is much faster and uses *much* less memory as it parses the input bytes in a streaming fashion via [aws-smithy-json](https://crates.io/crates/aws-smithy-json).\n\nRight now, it only outputs JSON paths (see below), but the backend is fully extendable, so any kind of output formatter can be written by implementing a trait.\n\nSee [src/path_value_writer/json_pointer.rs](https://github.com/ckampfe/jstream/blob/main/src/path_value_writer/json_pointer.rs) for what this looks like.\n\n## Installation\n\nLatest unstable (HEAD) release from source:\n\n```\n$ cargo install --git https://github.com/ckampfe/jstream\n```\n\n## Examples\n\nYou can pass JSON through stdin or a file (not shown):\n\n```\n$ echo '{\n  \"a\": 1,\n  \"b\": 2,\n  \"c\": [\"x\", \"y\", \"z\"],\n  \"d\": {\"e\": {\"f\": [{}, 9, \"g\"]}}\n}' | jstream    \n/a      1\n/b      2\n/c/0    \"x\"\n/c/1    \"y\"\n/c/2    \"z\"\n/d/e/f/1        9\n/d/e/f/2        \"g\"\n\n```\n\n## Command-line interface\n\n```\n$ jstream -h\nEnumerate the paths through a JSON document\n\nUsage: jstream [JSON_LOCATION]\n\nArguments:\n  [JSON_LOCATION]  A JSON file path\n\nOptions:\n  -h, --help     Print help\n  -V, --version  Print version\n```\n\n## Path output order\n\n`jstream` makes *no guarantees at all* about the order in which paths are output. Paths may appear depth-first, breadth-first, or any other order at all relative to their position in the input JSON document. Further, *any ordering is not guaranteed to be stable from one version to the next*,\nas it may change to aid the implementation of new optimizations.\nIf a stable order is important, I recommend using `sort` or some other after-the-fact mechanism, as the set of paths output from a given input document are guaranteed to be stable over time.\n\n## Output notes\n\nThe command line interface for `jstream` (`main.rs`) currently only outputs paths for singular JSON values (strings, numbers, booleans, and null). It does *not* emit paths for arrays or objects, even those that are empty. See above for an example of this behavior.\n\nThe library (`lib.rs`), however, is able to emit paths for empty arrays and empty objects.\n\nThis is subject to change.\n\n## Performance\n\nOn my machine, on a typical (large) payload, `jstream` runs at ~200MB/s.\n\n```\n$ /bin/ls -la ~/code/citylots.json\n-rw-rw-rw-@ 1 clark  staff  189778220 Jun 28  2021 /Users/clark/code/citylots.json\n\n$ hyperfine -w3 -r9 --output=null \"jstream ~/code/citylots.json\"\nBenchmark 1: jstream ~/code/citylots.json\n  Time (mean ± σ):     910.6 ms ±   2.4 ms    [User: 838.0 ms, System: 56.5 ms]\n  Range (min … max):   905.9 ms … 914.7 ms    9 runs\n```\n\nAnd `jstream` has barely any memory overhead compared to the size of the input due to its streaming nature. The following is on MacOS, so the maximum resident set size number of 191692800 is in bytes, meaning on this 181MB [citylots.json](https://github.com/zemirco/sf-city-lots-json/blob/master/citylots.json) input, the overhead is only ~1.8MB.\n\n```\nat [ 16:41:27 ] ➜ /usr/bin/time -l jstream ~/code/citylots.json \u003e/dev/null\n        0.96 real         0.84 user         0.07 sys\n           191692800  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n               11817  page reclaims\n                   2  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                   0  voluntary context switches\n                  32  involuntary context switches\n         14942138146  instructions retired\n          2917720211  cycles elapsed\n           191202560  peak memory footprint\n```\n\nTo run the included microbenchmarks:\n\n```\n# install the benchmark runner\n$ cargo install cargo-criterion\n```\n\n```\n# clone the project\n$ git clone https://github.com/ckampfe/jstream\n```\n\n```\n# run the benchmarks\n$ cd jstream\n$ cargo criterion\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fckampfe%2Fjstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fckampfe%2Fjstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fckampfe%2Fjstream/lists"}