{"id":16729171,"url":"https://github.com/keithduncan/json-projection","last_synced_at":"2025-09-23T19:15:09.643Z","repository":{"id":56879572,"uuid":"66217237","full_name":"keithduncan/json-projection","owner":"keithduncan","description":"Project a filtering transform over a JSON stream to process large JSON documents with low memory overhead.","archived":false,"fork":false,"pushed_at":"2019-02-08T11:05:02.000Z","size":778,"stargazers_count":8,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-28T05:04:30.123Z","etag":null,"topics":["json","schema","streaming-parser","transformer"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/keithduncan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-08-21T19:40:08.000Z","updated_at":"2020-06-01T08:16:50.000Z","dependencies_parsed_at":"2022-08-20T23:10:44.187Z","dependency_job_id":null,"html_url":"https://github.com/keithduncan/json-projection","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/keithduncan/json-projection","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keithduncan%2Fjson-projection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keithduncan%2Fjson-projection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keithduncan%2Fjson-projection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keithduncan%2Fjson-projection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/keithduncan","download_url":"https://codeload.github.com/keithduncan/json-projection/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keithduncan%2Fjson-projection/sbom","scorecard":{"id":554062,"data":{"date":"2025-08-11","repo":{"name":"github.com/keithduncan/json-projection","commit":"896a397a9d59b0ea5f1d4dee86f2bcef7e22b466"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.9,"checks":[{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":9,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Warn: project license file does not contain an FSF or OSI license."],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":9,"reason":"1 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-jppv-gw3r-w3q8"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-20T11:51:00.135Z","repository_id":56879572,"created_at":"2025-08-20T11:51:00.135Z","updated_at":"2025-08-20T11:51:00.135Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276632781,"owners_count":25676970,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-23T02:00:09.130Z","response_time":73,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","schema","streaming-parser","transformer"],"created_at":"2024-10-12T23:27:21.946Z","updated_at":"2025-09-23T19:15:09.606Z","avatar_url":"https://github.com/keithduncan.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# json-projection\n\nProject a filtering transform over a JSON stream to avoid loading large quantities\nof data into memory.\n\nThere are two parts to gem, `JsonProjection::Parser` which is initialised with a\nstream containing JSON data. Callers then request a stream of events to build up\nan object model or discard. This is akin to a Streaming XML Parser or SAX.\nThe parser is based on the state machine in [dgraham/json-stream/lib/json/stream/parser.rb](https://github.com/dgraham/json-stream/blob/master/lib/json/stream/parser.rb)\nmodified to support pulling events instead of having them pushed as the data is\npiped in.\n\nThe second part, `JsonProjection::Projector`, is also initialised with a stream\ncontaining JSON data. Internally it constructs a parser to pull events from.\nGiven a schema of data you are interested in `Projector` will pull events and\nconstruct the document subtrees for the portions of the document you are interested\nin, ignore the subtrees you aren't interested in.\n\nFor improved performance and lower memory requirements this algorithm is now built-in\nto [brianmario/yajl-ruby](https://github.com/brianmario/yajl-ruby/pull/171) as `Yajl::Projector`.\n\n## Examples\n\nGiven the following JSON data, imagine it continuing with many more keys you\ndon't need:\n\n```json\n[\n  {\n    \"user\": {\n      \"name\": \"keith\",\n      \"age\": 26,\n      \"jobs\": [\n        {\n          \"title\": \"director of overworking\",\n          \"company\": \"south coast software\",\n          \"department\": \"most\"\n        },\n        {\n          \"title\": \"some kind of computering\",\n          \"company\": \"github the website dot com\",\n          \"department\": true\n        }\n      ]\n    },\n    \"another key\": {\n\n    },\n    \"woah this document is huge\": {\n\n    },\n    \"many megabytes\": {\n\n    },\n    \"etc\": {\n\n    }\n  }\n]\n```\n\nWe only need the `user` key, and really just the the `user`'s `name` key and\neach `jobs`'s `title` key. We could implementing a schema-specific streaming\nevent handler for this data to ignore the keys we don't want and generate an\nobject model for the ones we do. That's a pain to build and maintain, what we\nreally want is a projection over the data just like we do when\n`SELECT list, of, keys FROM table`.\n\n```ruby\nstream = File.open(\"somewhere.json\", \"r\")\n\nprojector = JsonProjection::Projector.new(stream)\n\nschema = {\n  \"user\" =\u003e {\n    \"name\" =\u003e nil,\n    \"jobs\" =\u003e {\n      \"title\" =\u003e nil,\n    },\n  },\n}\ndata = projector.project(schema)\n\n# data = [\n# {\n#   \"user\" =\u003e {\n#     \"name\" =\u003e \"keith\",\n#     \"jobs\" =\u003e [\n#       { \"title\" =\u003e \"director of overworking\" },\n#       { \"title\" =\u003e \"some kind of computering\" },\n#     ]\n#   }\n# }\n# ]\n#\n# use data...\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkeithduncan%2Fjson-projection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkeithduncan%2Fjson-projection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkeithduncan%2Fjson-projection/lists"}