{"id":13817420,"url":"https://github.com/bcicen/jstream","last_synced_at":"2025-05-14T19:05:18.052Z","repository":{"id":57479983,"uuid":"138641672","full_name":"bcicen/jstream","owner":"bcicen","description":"Streaming JSON parser for Go","archived":false,"fork":false,"pushed_at":"2023-10-23T07:21:37.000Z","size":73,"stargazers_count":578,"open_issues_count":4,"forks_count":38,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-04-06T08:09:26.069Z","etag":null,"topics":["go","json","json-decoding","json-deserialization","json-parser","json-parsing-library"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bcicen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-06-25T19:39:12.000Z","updated_at":"2025-03-28T16:16:42.000Z","dependencies_parsed_at":"2024-01-15T16:11:39.992Z","dependency_job_id":"1dd34056-b946-4999-badd-16df2e0281a1","html_url":"https://github.com/bcicen/jstream","commit_stats":{"total_commits":35,"total_committers":3,"mean_commits":"11.666666666666666","dds":0.1428571428571429,"last_synced_commit":"6894471b8cb6e2df3585f643b1c39e12e154d4b2"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcicen%2Fjstream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcicen%2Fjstream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcicen%2Fjstream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcicen%2Fjstream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bcicen","download_url":"https://codeload.github.com/bcicen/jstream/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248710404,"owners_count":21149185,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","json","json-decoding","json-deserialization","json-parser","json-parsing-library"],"created_at":"2024-08-04T06:00:42.842Z","updated_at":"2025-04-13T11:45:36.923Z","avatar_url":"https://github.com/bcicen.png","language":"Go","readme":"\u003cp align=\"center\"\u003e\u003cimg width=\"350px\" src=\"jstream.png\" alt=\"jstream\"/\u003e\u003c/p\u003e\n\n#\n\n[![GoDoc](https://godoc.org/github.com/bcicen/jstream?status.svg)](https://godoc.org/github.com/bcicen/jstream)\n\n\n`jstream` is a streaming JSON parser and value extraction library for Go.\n\nUnlike most JSON parsers, `jstream` is document position- and depth-aware -- this enables the extraction of values at a specified depth, eliminating the overhead of allocating encompassing arrays or objects; e.g:\n\nUsing the below example document:\n\u003cimg width=\"85%\" src=\"https://bradley.codes/static/img/jstream-levels.gif\" alt=\"jstream\"/\u003e\n\nwe can choose to extract and act only the objects within the top-level array:\n```go\nf, _ := os.Open(\"input.json\")\ndecoder := jstream.NewDecoder(f, 1) // extract JSON values at a depth level of 1\nfor mv := range decoder.Stream() {\n  fmt.Printf(\"%v\\n \", mv.Value)\n}\n```\n\noutput:\n```\nmap[desc:RGB colors:[red green blue]]\nmap[desc:CMYK colors:[cyan magenta yellow black]]\n```\n\nlikewise, increasing depth level to `3` yields:\n```\nred\ngreen\nblue\ncyan\nmagenta\nyellow\nblack\n```\n\noptionally, kev:value pairs can be emitted as an individual struct:\n```go\ndecoder := jstream.NewDecoder(f, 2).EmitKV() // enable KV streaming at a depth level of 2\n```\n\n```\njstream.KV{desc RGB}\njstream.KV{colors [red green blue]}\njstream.KV{desc CMYK}\njstream.KV{colors [cyan magenta yellow black]}\n```\n\n## Installing \n\n```bash\ngo get github.com/bcicen/jstream\n```\n\n## Commandline\n\n`jstream` comes with a cli tool for quick viewing of parsed values from JSON input:\n\n```bash\njstream -d 1 \u003c input.json\n```\n\n```json\n{\"colors\":[\"red\",\"green\",\"blue\"],\"desc\":\"RGB\"}\n{\"colors\":[\"cyan\",\"magenta\",\"yellow\",\"black\"],\"desc\":\"CMYK\"}\n```\n\ndetailed output with `-v` option:\n```bash\ncat input.json | jstream -v -d -1\n\ndepth\tstart\tend\ttype   | value\n2\t018\t023\tstring | \"RGB\"\n3\t041\t046\tstring | \"red\"\n3\t048\t055\tstring | \"green\"\n3\t057\t063\tstring | \"blue\"\n2\t039\t065\tarray  | [\"red\",\"green\",\"blue\"]\n1\t004\t069\tobject | {\"colors\":[\"red\",\"green\",\"blue\"],\"desc\":\"RGB\"}\n2\t087\t093\tstring | \"CMYK\"\n3\t111\t117\tstring | \"cyan\"\n3\t119\t128\tstring | \"magenta\"\n3\t130\t138\tstring | \"yellow\"\n3\t140\t147\tstring | \"black\"\n2\t109\t149\tarray  | [\"cyan\",\"magenta\",\"yellow\",\"black\"]\n1\t073\t153\tobject | {\"colors\":[\"cyan\",\"magenta\",\"yellow\",\"black\"],\"desc\":\"CMYK\"}\n0\t000\t155\tarray  | [{\"colors\":[\"red\",\"green\",\"blue\"],\"desc\":\"RGB\"},{\"colors\":[\"cyan\",\"magenta\",\"yellow\",\"black\"],\"desc\":\"CMYK\"}]\n```\n\n### Options\n\nOpt | Description\n--- | ---\n-d \\\u003cn\\\u003e | emit values at depth n. if n \u003c 0, all values will be emitted\n-kv | output inner key value pairs as newly formed objects\n-v | output depth and offset details for each value\n-h | display help dialog\n\n## Benchmarks\n\nObligatory benchmarks performed on files with arrays of objects, where the decoded objects are to be extracted.\n\nTwo file sizes are used -- regular (1.6mb, 1000 objects) and large (128mb, 100000 objects)\n\ninput size | lib | MB/s | Allocated\n--- | --- | --- | ---\nregular | standard | 97 | 3.6MB\nregular | jstream | 175 | 2.1MB\nlarge | standard | 92 | 305MB\nlarge | jstream | 404 | 69MB\n\nIn a real world scenario, including initialization and reader overhead from varying blob sizes, performance can be expected as below:\n\u003cimg src=\"https://bradley.codes/static/img/bench.svg\" alt=\"jstream\"/\u003e\n","funding_links":[],"categories":["Misc","Go"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbcicen%2Fjstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbcicen%2Fjstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbcicen%2Fjstream/lists"}