{"id":21740138,"url":"https://github.com/bserdar/jsonstream","last_synced_at":"2025-04-13T03:40:31.556Z","repository":{"id":67456102,"uuid":"178595753","full_name":"bserdar/jsonstream","owner":"bserdar","description":"Go library to deal with streaming JSON documents with line-delimited JSON (NDJSON), record separator-delimited JSON, concatenated JSON, and length-prefixed JSON","archived":false,"fork":false,"pushed_at":"2019-04-28T03:24:08.000Z","size":15,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-26T20:51:23.337Z","etag":null,"topics":["go","json","jsonstream","ndjson"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bserdar.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-30T18:22:15.000Z","updated_at":"2025-03-26T06:35:01.000Z","dependencies_parsed_at":"2023-06-18T04:03:09.258Z","dependency_job_id":null,"html_url":"https://github.com/bserdar/jsonstream","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bserdar%2Fjsonstream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bserdar%2Fjsonstream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bserdar%2Fjsonstream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bserdar%2Fjsonstream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bserdar","download_url":"https://codeload.github.com/bserdar/jsonstream/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248660594,"owners_count":21141323,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","json","jsonstream","ndjson"],"created_at":"2024-11-26T06:12:16.841Z","updated_at":"2025-04-13T03:40:31.548Z","avatar_url":"https://github.com/bserdar.png","language":"Go","readme":"# JSON Streams\n\n[![GoDoc](https://godoc.org/github.com/bserdar/jsonstream?status.svg)](https://godoc.org/github.com/bserdar/jsonstream)\n\nThis library supports streaming JSON streaming conventions described\nin this [Wikipedia page](https://en.wikipedia.org/wiki/JSON_streaming).\n\nThis is for the concatenated JSON streams, where each JSON document is\nconcatenated one after the other:\n\n```\nrd:=jsonstream.NewConcatReader(reader)\nvar entry MyEntry\nfor {\n  err:=rd.Unmarshal(\u0026entry)\n  if err==io.EOF {\n    break\n  }\n  if err!=nil {\n    return err\n  }\n  processEntry(entry)\n}\n```\n\nThis is for JSON streams with a separate JSON document in each line\n(NDJSON):\n\n```\nndLinesReader:=jsonstream.NewLineReader(reader)\nndLinesWriter:=jsonstream.NewLineWriter(writer)\n```\n\nThis is for JSON streams separated with record separator delimiter:\n\n```\nseqReader:=jsonstream.NewSeqReader(reader) // 0x1e record separator\nseqWriter:=jsonstream.NewSeqWriter(writer) \n```\n```\nseqReader:=jsonstream.NewSeqReaderWithSep(reader,'\\n') // Line-separated JSON\nseqWriter:=jsonstream.NewSeqWriterWithSep(writer,'\\n')\n```\n\nThis is for JSON streams with length prefixed JSON stream, where each\nJSON document is prefixed by its byte length:\n\n```\nlpReader:=jsonstream.NewLenPrefixedReader(reader)\nlpWriter:=jsonstream.NewLenPrefixesWriter(writer)\n```\n## APIs\n\nAll four stream readers/writers use the same APIs.\n\n### Readers\n\n```\ndata, err:=reader.ReadRaw()\n```\n\nReadRaw reads the next JSON document. Only the ConcatReader validates\nthat the JSON document is a valid document, the remaining readers\nsimply read until the next delimiter. The returned byte array is a\nnewly allocated copy of the underlying read buffer. Some of the\nreaders use buffered input, so the state of the underlying reader is\nunknown.\n\n```\nvar data myStruct\nerr:=reader.Unmarshal(\u0026data)\n```\n\nUnmarshals the next entry from the input. For concatenated JSON,\nerrors invalidate the rest of the stream. For others stream processing\ncan continue.\n\n\n### Writers\n\n```\nerr:=writer.WriteRaw(data)\n```\n\nWriteRaw simply writes the []byte data to the output, with the correct\ndelimiter. For NDJSON, WriteRaw removes the newline characters from\ndata.\n\n```\nerr:=writer.Marshal(data)\n```\n\nMarshal first encodes data to JSON, and then writes it to the output.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbserdar%2Fjsonstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbserdar%2Fjsonstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbserdar%2Fjsonstream/lists"}