{"id":24645388,"url":"https://github.com/simonjang/s3-query-json","last_synced_at":"2026-05-02T11:32:15.196Z","repository":{"id":33145538,"uuid":"150324000","full_name":"SimonJang/s3-query-json","owner":"SimonJang","description":"Query JSON documents on S3 with SQL","archived":false,"fork":false,"pushed_at":"2023-01-05T03:09:22.000Z","size":339,"stargazers_count":1,"open_issues_count":12,"forks_count":0,"subscribers_count":1,"default_branch":"development","last_synced_at":"2026-02-15T08:42:08.468Z","etag":null,"topics":["datalake","nodejs","s3"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SimonJang.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-09-25T20:17:06.000Z","updated_at":"2023-03-04T04:45:16.000Z","dependencies_parsed_at":"2023-01-14T23:36:08.049Z","dependency_job_id":null,"html_url":"https://github.com/SimonJang/s3-query-json","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/SimonJang/s3-query-json","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SimonJang%2Fs3-query-json","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SimonJang%2Fs3-query-json/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SimonJang%2Fs3-query-json/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SimonJang%2Fs3-query-json/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SimonJang","download_url":"https://codeload.github.com/SimonJang/s3-query-json/tar.gz/refs/heads/development","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SimonJang%2Fs3-query-json/sbom","scorecard":{"id":130162,"data":{"date":"2025-08-11","repo":{"name":"github.com/SimonJang/s3-query-json","commit":"2c5f3bd7ce44c5e881f730962796888e5085f9a5"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.7,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/29 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":3,"reason":"dependency not pinned by hash detected -- score normalized to 3","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:16: update your workflow using https://app.stepsecurity.io/secureworkflow/SimonJang/s3-query-json/ci.yml/development?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:19: update your workflow using https://app.stepsecurity.io/secureworkflow/SimonJang/s3-query-json/ci.yml/development?enable=pin","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   1 out of   1 npmCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/ci.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'development'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":0,"reason":"39 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-968p-4wvh-cqc8","Warn: Project is vulnerable to: GHSA-67hx-6x53-jw92","Warn: Project is vulnerable to: GHSA-6chw-6frg-f759","Warn: Project is vulnerable to: GHSA-v88g-cgmw-v5xw","Warn: Project is vulnerable to: GHSA-93q8-gq69-wqmw","Warn: Project is vulnerable to: GHSA-rrc9-gqf8-8rwg","Warn: Project is vulnerable to: GHSA-v6h2-p8h4-qcjw","Warn: Project is vulnerable to: GHSA-grv7-fg5c-xmjg","Warn: Project is vulnerable to: GHSA-3xgq-45jj-v275","Warn: Project is vulnerable to: GHSA-gxpj-cx7g-858c","Warn: Project is vulnerable to: GHSA-w573-4hg7-7wgq","Warn: Project is vulnerable to: GHSA-ff7x-qrg7-qggm","Warn: Project is vulnerable to: GHSA-ww39-953v-wcq6","Warn: Project is vulnerable to: GHSA-pfrx-2q88-qq97","Warn: Project is vulnerable to: GHSA-765h-qjxv-5f44","Warn: Project is vulnerable to: GHSA-f2jv-r9rf-7988","Warn: Project is vulnerable to: GHSA-43f8-2h32-f4cj","Warn: Project is vulnerable to: GHSA-rc47-6667-2j5j","Warn: Project is vulnerable to: GHSA-qqgx-2p2h-9c37","Warn: Project is vulnerable to: GHSA-9c47-m6qq-7p4h","Warn: Project is vulnerable to: GHSA-6c8f-qphg-qjgp","Warn: Project is vulnerable to: GHSA-p6mc-m468-83gw","Warn: Project is vulnerable to: GHSA-29mw-wpgm-hmr9","Warn: Project is vulnerable to: GHSA-35jh-r3h4-6jhm","Warn: Project is vulnerable to: GHSA-952p-6rrq-rcjv","Warn: Project is vulnerable to: GHSA-f8q6-p94x-37v3","Warn: Project is vulnerable to: GHSA-vh95-rmgr-6w4m","Warn: Project is vulnerable to: GHSA-xvch-5gv4-984h","Warn: Project is vulnerable to: GHSA-px4h-xg32-q955","Warn: Project is vulnerable to: GHSA-hj48-42vr-x3v9","Warn: Project is vulnerable to: GHSA-9wv6-86v2-598j","Warn: Project is vulnerable to: GHSA-c2qf-rxjj-qqgw","Warn: Project is vulnerable to: GHSA-52f5-9888-hmc6","Warn: Project is vulnerable to: GHSA-7p7h-4mm5-852v","Warn: Project is vulnerable to: GHSA-38fc-wpqx-33j7","Warn: Project is vulnerable to: GHSA-j8xg-fqg3-53r7","Warn: Project is vulnerable to: GHSA-776f-qx25-q3cc","Warn: Project is vulnerable to: GHSA-c4w7-xm78-47vh","Warn: Project is vulnerable to: GHSA-p9pc-299p-vxgp"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-16T05:05:01.899Z","repository_id":33145538,"created_at":"2025-08-16T05:05:01.900Z","updated_at":"2025-08-16T05:05:01.900Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32533342,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-02T11:28:32.350Z","status":"ssl_error","status_checked_at":"2026-05-02T11:27:30.140Z","response_time":132,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datalake","nodejs","s3"],"created_at":"2025-01-25T14:15:00.609Z","updated_at":"2026-05-02T11:32:15.169Z","avatar_url":"https://github.com/SimonJang.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# s3-query-json [![Build Status](https://travis-ci.org/SimonJang/s3-query-json.svg?branch=master)](https://travis-ci.org/SimonJang/s3-query-json)\n\n\u003e Query JSON and NDJSON files on Amazon S3\n\n## About\n\nThe `selectObjectContent` [API](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#selectObjectContent-property) allows to easily query JSON and NDJSON data from S3. This node module provides a wrapper for this method, exposing the data as an aggregated result as a Promise.\n\nSee the official AWS [documentation](https://docs.aws.amazon.com/amazonglacier/latest/dev/s3-glacier-select-sql-reference-select.html) for more information.\n\n## Install\n\n```\n$ npm install s3-query-json\n```\n\n## Usage\n\n```js\nconst {query} = require('s3-query-json');\n\nquery('foobarbaz', 'users.ndjson', 'SELECT s.name FROM S3Object s')\n\t.then(data =\u003e console.log(data) // [{\"name\": \"unicorn\"}, {\"name\": \"rainbow\"}])\n\nquery('foobarbaz', 'users.ndjson', 'SELECT s.name FROM S3Object s', {documentType: 'NDJSON', compressionType: 'GZIP', stream: false, delimiter: '\\n', scanRange: {start: 1, end: 100}})\n\t.then(data =\u003e console.log(data) // [{\"name\": \"unicorn\"}, {\"name\": \"rainbow\"}])\n```\n\n## API\n\n### query(bucket, key, expression, [options])\n\n#### bucket\n\nType: `String`\n\nThe name of the bucket containing your object.\n\n#### key\n\nType: `String`\n\nThe key of the S3 object, either JSON or NDJSON format.\n\n#### expression\n\nType: `String`\n\nSQL query to be executed. Read the [documentation](https://docs.aws.amazon.com/amazonglacier/latest/dev/s3-glacier-select-sql-reference-select.html) on which SQL operations are allowed.\n\n#### options\n\nType: `Object`\n\nConfiguration object.\n\n##### [options.documentType]\n\nType: `String`\nDefault: `NDJSON`\n\nAccepts either `JSON` or `NDJSON` as input.\n\n##### [options.compressionType]\n\nType: `String`\nDefault: `NONE`\n\nCompression used in the S3 object. Either `NONE`, `GZIP` or `BZIP2`\n\n##### [options.stream]\n\nType: `Boolean`\nDefault: `false`\n\nFlag to indicate if the function returns a Promise with a original readstream. This allows more advanced custom processing.\n\n##### [options.delimiter]\n\nType: `string`\nDefault: `\\n`\n\nDelimiter with **a maximum length of 1** to separate the records that are being returned. To avoid difficult parsing, use a delimiter with no collision with the records returned.\n\n##### [options.scanRange]\n\nType: `Object`\nDefault: `undefined`\n\nThis property can only be used with non-compressed (ND)JSON files. Allows to scan a specified range of bytes. See the [documentation](https://docs.aws.amazon.com/AmazonS3/latest/API/API_SelectObjectContent.html#AmazonS3-SelectObjectContent-request-ScanRange) for more information. When `scanRange` is provided, atleast one of keys `start` or `end` must be provided.\n\n##### [options.scanRange.start]\n\nType: `Number`\n\nInclusive start of the range. Starts from 0\n\n##### [options.scanRange.end]\n\nType: `Number`\n\nInclusive end of the range. This is allowed to be out of bound of the object size.\n\n\n## License\n\nMIT © [Simon Jang](https://github.com/SimonJang)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonjang%2Fs3-query-json","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimonjang%2Fs3-query-json","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonjang%2Fs3-query-json/lists"}