{"id":13908125,"url":"https://github.com/lmammino/s3st","last_synced_at":"2025-09-20T07:39:52.540Z","repository":{"id":34782251,"uuid":"183636081","full_name":"lmammino/s3st","owner":"lmammino","description":"A command line utility that allows you to stream data from multiple S3 objects directly into your terminal","archived":false,"fork":false,"pushed_at":"2023-01-07T19:02:31.000Z","size":1597,"stargazers_count":76,"open_issues_count":11,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-09-05T03:34:53.155Z","etag":null,"topics":["aws","cloudtrail","command-line","logs","s3","s3-bucket","s3-storage","streams"],"latest_commit_sha":null,"homepage":"https://loige.co","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lmammino.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-26T13:45:06.000Z","updated_at":"2025-01-10T13:59:06.000Z","dependencies_parsed_at":"2023-01-15T09:08:35.531Z","dependency_job_id":null,"html_url":"https://github.com/lmammino/s3st","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/lmammino/s3st","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmammino%2Fs3st","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmammino%2Fs3st/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmammino%2Fs3st/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmammino%2Fs3st/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lmammino","download_url":"https://codeload.github.com/lmammino/s3st/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmammino%2Fs3st/sbom","scorecard":{"id":595990,"data":{"date":"2025-08-11","repo":{"name":"github.com/lmammino/s3st","commit":"18f688929111c3770b54886493ac231c3f04fbdc"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":1.5,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/18 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact 1.1.0 not signed: https://api.github.com/repos/lmammino/s3st/releases/27456037","Warn: release artifact 1.0.1 not signed: https://api.github.com/repos/lmammino/s3st/releases/20503545","Warn: release artifact 1.0.0 not signed: https://api.github.com/repos/lmammino/s3st/releases/20498295","Warn: release artifact 0.0.4 not signed: https://api.github.com/repos/lmammino/s3st/releases/17018519","Warn: release artifact 0.0.3 not signed: https://api.github.com/repos/lmammino/s3st/releases/17018296","Warn: release artifact 1.1.0 does not have provenance: https://api.github.com/repos/lmammino/s3st/releases/27456037","Warn: release artifact 1.0.1 does not have provenance: https://api.github.com/repos/lmammino/s3st/releases/20503545","Warn: release artifact 1.0.0 does not have provenance: https://api.github.com/repos/lmammino/s3st/releases/20498295","Warn: release artifact 0.0.4 does not have provenance: https://api.github.com/repos/lmammino/s3st/releases/17018519","Warn: release artifact 0.0.3 does not have provenance: https://api.github.com/repos/lmammino/s3st/releases/17018296"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 3 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":0,"reason":"32 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-v88g-cgmw-v5xw","Warn: Project is vulnerable to: GHSA-93q8-gq69-wqmw","Warn: Project is vulnerable to: GHSA-rrc9-gqf8-8rwg","Warn: Project is vulnerable to: GHSA-pp7h-53gx-mx7r","Warn: Project is vulnerable to: GHSA-v6h2-p8h4-qcjw","Warn: Project is vulnerable to: GHSA-grv7-fg5c-xmjg","Warn: Project is vulnerable to: GHSA-3xgq-45jj-v275","Warn: Project is vulnerable to: GHSA-gxpj-cx7g-858c","Warn: Project is vulnerable to: GHSA-w573-4hg7-7wgq","Warn: Project is vulnerable to: GHSA-fjxv-7rqg-78g4","Warn: Project is vulnerable to: GHSA-ww39-953v-wcq6","Warn: Project is vulnerable to: GHSA-pfrx-2q88-qq97","Warn: Project is vulnerable to: GHSA-43f8-2h32-f4cj","Warn: Project is vulnerable to: GHSA-rc47-6667-2j5j","Warn: Project is vulnerable to: GHSA-qqgx-2p2h-9c37","Warn: Project is vulnerable to: GHSA-896r-f27r-55mw","Warn: Project is vulnerable to: GHSA-9c47-m6qq-7p4h","Warn: Project is vulnerable to: GHSA-p6mc-m468-83gw","Warn: Project is vulnerable to: GHSA-29mw-wpgm-hmr9","Warn: Project is vulnerable to: GHSA-35jh-r3h4-6jhm","Warn: Project is vulnerable to: GHSA-952p-6rrq-rcjv","Warn: Project is vulnerable to: GHSA-f8q6-p94x-37v3","Warn: Project is vulnerable to: GHSA-xvch-5gv4-984h","Warn: Project is vulnerable to: GHSA-px4h-xg32-q955","Warn: Project is vulnerable to: GHSA-hj48-42vr-x3v9","Warn: Project is vulnerable to: GHSA-hrpp-h998-j3pp","Warn: Project is vulnerable to: GHSA-p8p7-x288-28g6","Warn: Project is vulnerable to: GHSA-c2qf-rxjj-qqgw","Warn: Project is vulnerable to: GHSA-52f5-9888-hmc6","Warn: Project is vulnerable to: GHSA-72xf-g2v4-qvf3","Warn: Project is vulnerable to: GHSA-j8xg-fqg3-53r7","Warn: Project is vulnerable to: GHSA-776f-qx25-q3cc"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-20T23:09:42.438Z","repository_id":34782251,"created_at":"2025-08-20T23:09:42.438Z","updated_at":"2025-08-20T23:09:42.438Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276064303,"owners_count":25578998,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-20T02:00:10.207Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","cloudtrail","command-line","logs","s3","s3-bucket","s3-storage","streams"],"created_at":"2024-08-06T23:02:29.101Z","updated_at":"2025-09-20T07:39:52.501Z","avatar_url":"https://github.com/lmammino.png","language":"JavaScript","funding_links":[],"categories":["HarmonyOS"],"sub_categories":["Windows Manager"],"readme":"# s3st\n\nA command line utility that allows you to stream data from multiple S3 objects\ndirectly into your terminal.\n\n[![npm version](https://badge.fury.io/js/s3st.svg)](https://badge.fury.io/js/s3st)\n[![CircleCI](https://circleci.com/gh/lmammino/s3st.svg?style=shield)](https://circleci.com/gh/lmammino/s3st)\n[![JavaScript Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://standardjs.com)\n\n## Demo!\n\n[![Demo image terminal](s3st.gif)](https://asciinema.org/a/dWJtrXA0HRqDJxndId9Xauz0e)\n\n[See the FULL demo on asciinema](https://asciinema.org/a/dWJtrXA0HRqDJxndId9Xauz0e)\n\n## Rationale\n\nThis utility is particularly useful when you are storing data in S3 and you want\nto easily process the content of your S3 objects from your command line,\nfor instance if you are storing your CloudTrail logs in an S3 buckets and you\nwant to grep over them you can do something like this:\n\n```bash\ns3st mybucket AWSLogs/123456789/CloudTrail/eu-west-1/2019/01/17/ | jq . | grep \"lambda\"\n```\n\nBy default the command line will be able to decompress most compressed files in\nrealtime (gzip, brotli and deflate).\n\n\n## Install\n\nThere are several ways to install `s3st`:\n\n### Install global with NPM\n\n(Requires Node v10+):\n\n```bash\nnpm i -g s3st\n```\n\n### Precompiled binaries\n\nAlternatively you can download one of the pre-compiled binaries for linux,\nwindows, mac or alpine from the [Releases page](https://github.com/lmammino/s3st/releases).\n\nThese binaries do not require you to have Node installed.\n\n### With [npx](https://www.npmjs.com/package/npx) (use without install)\n\n```bash\nnpx s3st some-s3-bucket\n```\n\n\n## Usage\n\n```bash\nUsage: s3st [options] \u003cbucket\u003e [prefix]\n\nOptions:\n  -v, --version            output the version number\n  -D, --do-not-decompress  Do not try to decompress files automatically (gzip, deflate, brotli)\n  -h, --help               output usage information\n```\n\n`bucket` represents the name of the bucket to iterate over\n`prefix` is an optional argument that you can pass to select a subset of object\nthat match the given prefix.\n\n\n## Automatic Decompression\n\nThe command will automatically try to decompress compressed files based on their\nextension, as per the following mapping:\n\n - `.gz` or `.gzip`: decompress using gzip\n - `.zz` or `.deflate`: decompress using deflate\n - `.br` or `.brotli`: decompress using brotli (available only if using Node v11.7+)\n\nIf you want to disable this option you can specify the flag `--do-not-decompress`\n\n\n## AWS Authentication\n\nThe tool will assume you have the proper environment variables or configuration\nfiles properly set as per the [AWS CLI documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html)\nin order to authenticate requests to AWS.\n\n\n## Programmatic usage\n\nThis package can also be used programmatically as per the following example:\n\n```javascript\n'use strict'\n\nconst createS3stStream = require('s3st')\nconst AWS = require('aws-sdk')\n\n// creates an s3 client using the AWS SDK\nconst s3 = new AWS.S3()\n\nconst stream = createS3stStream(s3, 'mybucket', 'some-prefix')\n\nstream.pipe(process.stdout) // attach the stream to standard output\n```\n\n`createS3stStream` exposes accepts the following arguments:\n\n- `s3`: an s3 client instance from the AWS SDK or a compatible implementation\n- `bucketName`: the name of the bucket\n- `prefix` (optional): an object prefix to filter objects in the bucket\n- `transform` (optional): a function that allows you to transform the content of\nobjects as they get streamed (useful for instance for decompression or decryption).\n\n### Transform function\n\nIf you want to provide a custom transform function, it should respect the following\nsignature.\n\n#### Arguments\n - `key` (string): the name of the current object (object key)\n\n#### Return value\n - a `Transform` stream that manipulates the object\n\nIf you want to use the default decompression implementation available by the\ndefault in the command line client, you can import that from [`s3st/src/transformers/decompress`](/src/transformers/decompress.js).\n\n\n## Data Transfer costs\n\nIf you are using this tool to stream large amount of data be aware that this might have an impact on your [data transfer costs](https://blog.cloudability.com/aws-data-transfer-costs/). In such cases, using an alternative approach like [S3 Select](https://docs.aws.amazon.com/AmazonS3/latest/dev/selecting-content-from-objects.html), could be a way to save on cost.\n\nMake sure you are aware of alternatives and that you make careful costs considerations before running any heavy workload in the cloud.\n\n\n## Contributing\n\nEveryone is very welcome to contribute to this project. You can contribute just by submitting bugs or\nsuggesting improvements by [opening an issue on GitHub](https://github.com/lmammino/s3st/issues).\n\nYou can also submit PRs as long as you adhere with the code standards and write tests for the proposed changes.\n\n## License\n\nLicensed under [MIT License](LICENSE). © Luciano Mammino.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmammino%2Fs3st","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flmammino%2Fs3st","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmammino%2Fs3st/lists"}