{"id":15400107,"url":"https://github.com/indutny/json-depth-stream","last_synced_at":"2025-04-15T22:30:37.265Z","repository":{"id":9837556,"uuid":"63503549","full_name":"indutny/json-depth-stream","owner":"indutny","description":"Streaming JSON parser with depth-limited auxiliary data","archived":false,"fork":false,"pushed_at":"2022-12-30T19:33:39.000Z","size":254,"stargazers_count":43,"open_issues_count":8,"forks_count":6,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-29T03:12:15.575Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/indutny.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-07-16T21:49:03.000Z","updated_at":"2024-12-16T01:54:53.000Z","dependencies_parsed_at":"2023-01-13T15:35:33.221Z","dependency_job_id":null,"html_url":"https://github.com/indutny/json-depth-stream","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indutny%2Fjson-depth-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indutny%2Fjson-depth-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indutny%2Fjson-depth-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indutny%2Fjson-depth-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/indutny","download_url":"https://codeload.github.com/indutny/json-depth-stream/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249165833,"owners_count":21223330,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T15:51:52.312Z","updated_at":"2025-04-15T22:30:37.013Z","avatar_url":"https://github.com/indutny.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# json-depth-stream\n\n[![NPM version](https://badge.fury.io/js/json-depth-stream.svg)](http://badge.fury.io/js/json-depth-stream)\n[![Build Status](https://secure.travis-ci.org/indutny/json-depth-stream.svg)](http://travis-ci.org/indutny/json-depth-stream)\n\n## Why?\n\nIt is impossible to parse 1gb JSON file with native node.js primitives.\n\n## How?\n\nJust a streaming parser, nothing really fancy. The only difference between this\nparser and the others is that it can skip data that is nested too deeply.\n\n## Installation\n\n```bash\nnpm install json-depth-stream\n```\n\n## Usage\n\n\n```js\nconst DepthStream = require('json-depth-stream');\n\nconst file = require('fs').createReadStream('/tmp/big.json');\nconst json = new DepthStream(1 /* desired depth */);\n\njson.on('visit', (path, start, end) =\u003e {\n  // `path` is a nested JSON key like: `[\"a\", 0, \"b\"]`, which is equivalent to\n  // `obj.a[0].b` (assuming `obj.a` is an Array)\n\n  // `start` and `end` are offsets in the input stream\n  // they could be used in `fs.read()` if streaming from file.\n  console.log(path, start, end);\n});\n\nfile.pipe(json);\n```\n\nQuery:\n```js\nconst json = new DepthStream(1);\n\nconst q = json.query([ 'path' ]);\nq.pipe(process.stdout);\nq.once('end', () =\u003e {\n  process.stdin.unpipe(json);\n});\n\nprocess.stdin.pipe(json);\n```\n\nStream-once usage:\n```js\nconst json = new DepthStream(1);\n\nprocess.stdin.on('data', (chunk) =\u003e {\n  json.update(chunk);\n});\n\n// `split` event is emitted when parser enters new object value/array element.\n// `index` is local to `chunk`.\njson.on('split', (path, index) =\u003e {\n  // `path` is the same as above\n  // `index` is an index in a `chunk` that was given to `.update(chunk)`\n  console.log(path, index);\n});\n```\n\n## Speed\n\nAround 100mb/s for JSON with many object keys, faster for array-like JSONs:\n\n```bash\n$ node benchmark/throughput.js\nThroughput: 100.00 mb/s\n```\n\n## LICENSE\n\nThis software is licensed under the MIT License.\n\nCopyright Fedor Indutny, 2020.\n\nPermission is hereby granted, free of charge, to any person obtaining a\ncopy of this software and associated documentation files (the\n\"Software\"), to deal in the Software without restriction, including\nwithout limitation the rights to use, copy, modify, merge, publish,\ndistribute, sublicense, and/or sell copies of the Software, and to permit\npersons to whom the Software is furnished to do so, subject to the\nfollowing conditions:\n\nThe above copyright notice and this permission notice shall be included\nin all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS\nOR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN\nNO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,\nDAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR\nOTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE\nUSE OR OTHER DEALINGS IN THE SOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Findutny%2Fjson-depth-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Findutny%2Fjson-depth-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Findutny%2Fjson-depth-stream/lists"}