{"id":16986054,"url":"https://github.com/chung-leong/progressive-json","last_synced_at":"2026-04-17T10:32:23.504Z","repository":{"id":65817251,"uuid":"596613339","full_name":"chung-leong/progressive-json","owner":"chung-leong","description":"Retrieve JSON progressively","archived":false,"fork":false,"pushed_at":"2023-02-27T07:04:42.000Z","size":6110,"stargazers_count":14,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-10-20T02:16:04.274Z","etag":null,"topics":["json","range-requests","react"],"latest_commit_sha":null,"homepage":"https://chung-leong.github.io/progressive-json/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chung-leong.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-02T15:11:54.000Z","updated_at":"2025-06-30T07:16:52.000Z","dependencies_parsed_at":"2023-03-10T19:09:45.563Z","dependency_job_id":null,"html_url":"https://github.com/chung-leong/progressive-json","commit_stats":{"total_commits":63,"total_committers":1,"mean_commits":63.0,"dds":0.0,"last_synced_commit":"36d9b92d898cc2b390af8a67d02a2550ebe5339a"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/chung-leong/progressive-json","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chung-leong%2Fprogressive-json","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chung-leong%2Fprogressive-json/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chung-leong%2Fprogressive-json/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chung-leong%2Fprogressive-json/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chung-leong","download_url":"https://codeload.github.com/chung-leong/progressive-json/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chung-leong%2Fprogressive-json/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31925396,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-17T10:19:20.377Z","status":"ssl_error","status_checked_at":"2026-04-17T10:19:18.682Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","range-requests","react"],"created_at":"2024-10-14T02:44:44.724Z","updated_at":"2026-04-17T10:32:23.488Z","avatar_url":"https://github.com/chung-leong.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Progressive-json ![ci](https://img.shields.io/github/actions/workflow/status/chung-leong/progressive-json/node.js.yml?branch=main\u0026label=Node.js%20CI\u0026logo=github) ![nycrc config on GitHub](https://img.shields.io/nycrc/chung-leong/progressive-json)\n\nA Javascript library that processes JSON data progressively. Instead of making you wait until\nthe file transfer has completed, it provides you with continual snapshots of the  object as\ndata arrives from the remote server: \n\n![Growing JSON](./doc/img/animation.gif)\n\nProgressive retrieval helps improve preceived load time. This library also lets you download a\nJSON file incrementally using \n[HTTP range requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests), \nmaking it practical to host data-driven app on static website providers like \n[GitHub Pages](https://pages.github.com/). \n\n## Installation\n\n```sh\nnpm  --save-dev install progressive-json\n```\n\n## Feature set\n\n* An async generator function for retrieving snapshots of JSON objects: [fetchJSON](./doc/fetchJSON.md)\n* React hooks for make use of progressively loaded JSON: [useProgressiveJSON](./doc/useProgressiveJSON.md) and [usePartialJSON](./doc/usePartialJSON.md)\n* A server-side function for streaming of large JSON objects: [createJSONStream](./doc/createJSONStream.md)\n\n## How it works\n\nProgress-JSON does not actually perform any JSON parsing on its own. All it does is monitoring the byte stream for certain characters:\n\n![Special characters](./doc/img/characters.jpg)\n\nShown in black are these special characters: right and left curlies, right and left brackets, comma, double quotation mark, and backslash. Shown with a green background are the locations where the JSON structure can be safely cleaved: immediately before a comma and immediately after a curly or square bracket.\n\nSupposed we want to stop parsing after the first item in `results`. We want to cleave the structure at the spot indicated by the arrow. To make the fragment syntactically correct, we just need to append `] }`. We know that because we've been monitoring the appearances of different brackets. To make the next fragment syntactically correctly, we need to prepend it with `{ \"\": [`. \n\nActual JSON parsing is done by \n[`JSON.parse()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse). \n\n## Partial JSON Demonstration\n\nProgressive-json does not make use of every possible cleave points. Only objects specified by you\nthrough the `partial` option will show up partially complete. The \n[project demo site](https://chung-leong.github.io/progressive-json/) shows you how this is done:\n\n![Initial view](./doc/img/screenshot-1.jpg)\n\nInitially, `partial` is set to \"results\". That means the root level object and object.results can be \npartial. If you change `partial` to \"results.#\", you'll see the following:\n\n![Partial objects](./doc/img/screenshot-2.jpg)\n\nNow items inside the array can be partial too (# in the object path indicates array item). If we want \nthe list of films in the character object to be partial, we would specify \"results.#.films\". If we \nwant all the lists to be partial, we would specify \"results.#.*\".\n\nThe chunk size is set to a unrealistic small size of 16 for demonstration purpose. Make it bigger (5K ~ 10K) to get a better sense of how an object would be divided in actual usage. \n\n## Examples\n\n* [USDA dood data example](./examples/food-data#readme) - A website using data from a large static JSON file\n* [WordPress example](./examples/wordpress#readme) - Streaming JSON from a MySQL database\n\n## API reference\n\n#### Client-side:\n\n* [fetchJSON](./doc/fetchJSON.md)\n* [generateJSON](./doc/generateJSON.md)\n* [getJSONProgress](./doc/getJSONProgress.md)\n* [useArraySlice](./doc/useArraySlice.md)\n* [usePartialJSON](./doc/usePartialJSON.md)\n* [useProgressiveJSON](./doc/useProgressiveJSON.md)\n\n#### Server-side:\n\nServer-side functions are found in `\"progressive-json/server\"`.\n\n* [createJSONStream](./doc/createJSONStream.md)\n* [countGenerated](./doc/countGenerated.md)\n* [deferred](./doc/deferred.md)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchung-leong%2Fprogressive-json","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchung-leong%2Fprogressive-json","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchung-leong%2Fprogressive-json/lists"}