{"id":18739052,"url":"https://github.com/gaubee/big-json-parser","last_synced_at":"2025-11-18T13:30:13.221Z","repository":{"id":149112060,"uuid":"360516262","full_name":"Gaubee/big-json-parser","owner":"Gaubee","description":null,"archived":false,"fork":false,"pushed_at":"2021-04-22T12:48:22.000Z","size":3520,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-12-28T17:44:53.018Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Gaubee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-22T12:46:02.000Z","updated_at":"2021-12-18T09:35:07.000Z","dependencies_parsed_at":"2023-04-26T01:02:29.223Z","dependency_job_id":null,"html_url":"https://github.com/Gaubee/big-json-parser","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gaubee%2Fbig-json-parser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gaubee%2Fbig-json-parser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gaubee%2Fbig-json-parser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gaubee%2Fbig-json-parser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Gaubee","download_url":"https://codeload.github.com/Gaubee/big-json-parser/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239616727,"owners_count":19669098,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T15:32:32.737Z","updated_at":"2025-11-18T13:30:13.195Z","avatar_url":"https://github.com/Gaubee.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"这是一个学习项目。\n尝试使用 go 来编写一个基于 wasm 的 json-stream 。并对比对照代码写得比较好的纯 js 库的实现。\n\n因为目前 wasm 的特性，是很难超越“高性能 js”的实现。毕竟有一个 Bytes 转换的成本在那边。\n所以该项目的目的是研究 go-wasm，将性能尽可能逼近 pure-js。\n\n## TIP\n\n1. 这个项目对 golang 源码进行了一些改动，一部分是改进不成熟的 wasm 相关的实现。还有一部分是对官方 encoding/json 的改动\n\n   \u003e 项目地址： https://github.com/Gaubee/go\n   \u003e commit:736a02826389faf431dba2cab47b64f8fef307bc\n\n   1. encoding/json 是官方提供的流处理 json 的实现，出了名的性能差，但至少能用。要对其进行改动是因为 wasm 目前不完全支持 sync，所以对于数据流中的暂停没有支持，因此只能做一些魔改\n   2. wasm 相关的改动主要是两部分：性能提升与自定义异常抛出。\n      1. go-wasm 中直接抛出异常是直接结束整个 GoRuntime，所以我定义了一个 jsRuntimeError，来模拟 js-throw 的特性。\n      1. 性能方面，主要是在 Bytes 转 Utf8 的实现。TextDecoder 很快，但在处理少量数据时（大概少于 200），会比 pure-js 实现来得慢，猜测是 js 与 native 之间的转换成本。毕竟每一次调用 native 都有一个基础的调用成本，\n\n2. 项目用到了 typescript 来编写 js 部分的工作。\n3. 用 esbuild 来将 js 与 wasm 两部分的文件进行组合打包\n4. 用 wmr 来提供文件服务（可以实时处理 typescript-esm 的输出）\n5. jsonreader.js 这个文件是 nodejs 程序，提供一个动态的 json 数据包。数据包模拟了区块链数据。\n   \u003e http://localhost:8301/?s=10\u0026e=20\n   1. **s** 为 startHeigth\n   1. **e** 为 endHeigth\n\n## Benchmark\n\n解析 200_0000 条数据：下载数据+解析数据\n\n1. 直接使用 fetch.response.json() 来作为最基准的性能\n   1. 浏览器~= 6.1560s\n   1. nodejs~= 2.5330s\n1. 使用流处理的模式来解析数据\n   1. browser/wasm/go-json-stream ~= 23.3318s\n   1. nodejs/js/JSONStream ~= 8.3680s\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgaubee%2Fbig-json-parser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgaubee%2Fbig-json-parser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgaubee%2Fbig-json-parser/lists"}