{"id":21344615,"url":"https://github.com/promplate/partial-json-parser","last_synced_at":"2025-04-12T22:20:24.036Z","repository":{"id":200810512,"uuid":"706085025","full_name":"promplate/partial-json-parser","owner":"promplate","description":"Parse partial JSON generated by LLM","archived":false,"fork":false,"pushed_at":"2025-01-08T18:23:58.000Z","size":43,"stargazers_count":64,"open_issues_count":2,"forks_count":4,"subscribers_count":2,"default_branch":"with-action","last_synced_at":"2025-04-04T03:07:20.752Z","etag":null,"topics":["agent","json","langchain","llm","parser","prompt-engineering","streaming"],"latest_commit_sha":null,"homepage":"https://promplate.dev/partial-json-parser","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/promplate.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-17T09:21:09.000Z","updated_at":"2025-04-01T06:43:29.000Z","dependencies_parsed_at":"2025-03-28T03:00:58.304Z","dependency_job_id":"627ec848-eacc-406b-8421-86dbac38b859","html_url":"https://github.com/promplate/partial-json-parser","commit_stats":null,"previous_names":["promplate/partial-json-parser"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promplate%2Fpartial-json-parser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promplate%2Fpartial-json-parser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promplate%2Fpartial-json-parser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promplate%2Fpartial-json-parser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/promplate","download_url":"https://codeload.github.com/promplate/partial-json-parser/tar.gz/refs/heads/with-action","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248638400,"owners_count":21137660,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","json","langchain","llm","parser","prompt-engineering","streaming"],"created_at":"2024-11-22T01:20:37.083Z","updated_at":"2025-04-12T22:20:24.014Z","avatar_url":"https://github.com/promplate.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Partial JSON Parser\n\nSometimes we need **LLM (Large Language Models)** to produce **structural information** instead of natural language. The easiest way is to use JSON.\n\nBut before receiving the last token of response, the JSON is broken, which means you can't use `JSON.parse` to decode it. But we still want to stream the data to the user.\n\nHere comes `partial-json-parser`, a lightweight and customizable library for parsing partial JSON strings. Here is a [demo](https://promplate.dev/partial-json-parser).\n\n(Note that there is [a JavaScript implementation](https://github.com/promplate/partial-json-parser-js) too)\n\n## Installation\n\n```sh\npip install partial-json-parser # or poetry / pdm / uv\n```\n\n`partial-json-parser` is implemented purely in Python, with good type hints. It is zero-dependency and works with Python 3.6+.\n\nYou can install run its demo playground by installing `rich` too or:\n\n```sh\npip install partial-json-parser[playground]\n```\n\nThen run the `json-playground` in your terminal, and you can try the parser interactively.\n\n## Usage\n\n```py\nfrom partial_json_parser import loads\n\n\u003e\u003e\u003e loads('{\"key\": \"v')  # {'key': 'v'}\n```\n\nAlternatively, you can use `ensure_json` to get the completed JSON string:\n\n```py\nfrom partial_json_parser import ensure_json\n\n\u003e\u003e\u003e ensure_json('{\"key\": \"v')  # '{\"key\": \"v\"}'\n```\n\n### Detailed Usage\n\nYou can import the `loads` function and the `Allow` object from the library like this:\n\n```py\nfrom partial_json_parser import loads, Allow\n```\n\nThe `Allow` object is just an Enum for options. It determines what types can be partial. types not included in `allow` only appears after its completion can be ensured.\n\n### Parsing complete / partial JSON strings\n\nThe `loads` function works just like the built-in `json.loads` when parsing a complete JSON string:\n\n```py\nresult = loads('{\"key\":\"value\"}')\nprint(result)  # Outputs: {'key': 'value'}\n```\n\nYou can parse a partial JSON string by passing an additional parameter to the `loads` function. This parameter is a **bitwise OR** of the constants from the `Allow` flag:\n\n(Note that you can directly import the constants you need from `partial-json-parser`)\n\n```py\nfrom partial_json_parser import loads, Allow, STR, OBJ\n\nresult = loads('{\"key\": \"v', STR | OBJ)\nprint(result)  # Outputs: {'key': 'v'}\n```\n\nIn this example, `Allow.STR` tells the parser that it's okay if a string is incomplete, and `Allow.OBJ` tells the parser so as a dict. The parser then try to return as much data as it can.\n\nIf you don't allow partial strings, then it will not add `\"key\"` to the object because `\"v` is not close:\n\n```py\nresult = loads('{\"key\": \"v', OBJ)\nprint(result)  # Outputs: {}\n\nresult = loads('{\"key\": \"value\"', OBJ)\nprint(result)  # Outputs: {'key': 'value'}\n```\n\nSimilarity, you can parse partial lists or even partial special values if you allow it:\n\n(Note that `allow` defaults to `Allow.ALL`)\n\n```py\nresult = loads('[ {\"key1\": \"value1\", \"key2\": [ \"value2')\nprint(result)  # Outputs: [{'key1': 'value1', 'key2': ['value2']}]\n\nresult = loads(\"-Inf\")\nprint(result)  # Outputs: -inf\n```\n\n### Handling malformed JSON\n\nIf the JSON string is malformed, the `parse` function will throw an error:\n\n```py\nloads(\"wrong\")  # MalformedJSON: Malformed node or string on line 1\n```\n\n## API Reference\n\n### loads(json_string, [allow_partial], [parser])\n\n- `json_string` `\u003cstring\u003e`: The (incomplete) JSON string to parse.\n- `allow_partial` `\u003cAllow | int\u003e`: Specify what kind of partialness is allowed during JSON parsing (default: `Allow.ALL`).\n- `parser` `(str) -\u003e JSON`: An ordinary JSON parser. Default is `json.loads`.\n\nComplete the JSON string and parse it with `parser` function.\n\nReturns the parsed Python value.\n\nAlias: `decode`, `parse_json`.\n\n### ensure_json(json_string, [allow_partial])\n\n- `json_string` `\u003cstring\u003e`: The (incomplete) JSON string to complete.\n- `allow_partial` `\u003cAllow | int\u003e`: Specify what kind of partialness is allowed during JSON parsing (default: `Allow.ALL`).\n\nReturns the completed JSON string.\n\n### fix(json_string, [allow_partial])\n\n- `json_string` `\u003cstring\u003e`: The (incomplete) JSON string to complete.\n- `allow_partial` `\u003cAllow | int\u003e`: Specify what kind of partialness is allowed during JSON parsing (default: `Allow.ALL`).\n\nReturns a tuple of a slice of the input string and the completion.\n\nNote that this is a low-level API, only useful for debugging and demonstration.\n\n### Allow\n\nEnum class that specifies what kind of partialness is allowed during JSON parsing. It has the following members:\n\n- `STR`: Allow partial string.\n- `NUM`: Allow partial number.\n- `ARR`: Allow partial array.\n- `OBJ`: Allow partial object.\n- `NULL`: Allow partial null.\n- `BOOL`: Allow partial boolean.\n- `NAN`: Allow partial NaN.\n- `INFINITY`: Allow partial Infinity.\n- `_INFINITY`: Allow partial -Infinity.\n- `INF`: Allow both partial Infinity and -Infinity.\n- `SPECIAL`: Allow all special values.\n- `ATOM`: Allow all atomic values.\n- `COLLECTION`: Allow all collection values.\n- `ALL`: Allow all values.\n\n## Testing\n\nTo run the tests for this library, you should clone the repository and install the dependencies:\n\n```sh\ngit clone https://github.com/promplate/partial-json-parser.git\ncd partial-json-parser\npdm install\n```\n\nThen, you can run the tests using [Hypothesis](https://hypothesis.works/) and [Pytest](https://pytest.org/):\n\n```sh\npdm test\n```\n\nPlease note that while we strive to cover as many edge cases as possible, it's always possible that some cases might not be covered.\n\n## License\n\nThis project is licensed under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpromplate%2Fpartial-json-parser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpromplate%2Fpartial-json-parser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpromplate%2Fpartial-json-parser/lists"}