{"id":29417237,"url":"https://github.com/dacort/s3grep","last_synced_at":"2026-01-21T13:25:19.260Z","repository":{"id":303470850,"uuid":"1015566493","full_name":"dacort/s3grep","owner":"dacort","description":"CLI tool for searching logs and unstructured content in Amazon S3 buckets","archived":false,"fork":false,"pushed_at":"2025-07-08T01:09:31.000Z","size":95,"stargazers_count":213,"open_issues_count":1,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-01-19T22:48:39.447Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dacort.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-07T17:39:06.000Z","updated_at":"2026-01-16T18:07:07.000Z","dependencies_parsed_at":"2025-07-07T21:32:35.843Z","dependency_job_id":"f38fa7f3-1e2e-4e04-b916-886e4da6e311","html_url":"https://github.com/dacort/s3grep","commit_stats":null,"previous_names":["dacort/s3grep"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/dacort/s3grep","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacort%2Fs3grep","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacort%2Fs3grep/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacort%2Fs3grep/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacort%2Fs3grep/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dacort","download_url":"https://codeload.github.com/dacort/s3grep/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacort%2Fs3grep/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28633757,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-21T04:47:28.174Z","status":"ssl_error","status_checked_at":"2026-01-21T04:47:22.943Z","response_time":86,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-11T20:10:01.530Z","updated_at":"2026-01-21T13:25:19.244Z","avatar_url":"https://github.com/dacort.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"# s3grep - Fast Parallel grep for S3\n\nI use `grep` almost daily, but often deal with unstructured content and logs on S3...so I wrote an easy way to `grep` S3!\n\n[![CI](https://github.com/dacort/s3grep/actions/workflows/ci.yml/badge.svg)](https://github.com/dacort/s3grep/actions/workflows/ci.yml)\n\n`s3grep` is a parallel CLI tool for searching logs and unstructured content in Amazon S3 buckets. It supports .gz decompression, progress bars, and robust error handling—making it ideal for cloud-native log analysis.\n\n---\n\n## Features\n\n- Parallel, concurrent search across S3 objects\n- Supports plain text and `.gz` compressed files\n- Progress bars for files and bytes processed\n- Case-sensitive and insensitive search\n- Line number output option\n- Graceful handling of binary files and decompression errors\n- Colorized match highlighting\n\n---\n\n## Installation\n\n### crates.io\n\n```bash\ncargo install s3grep\n```\n\n---\n\n## Usage\n\n```sh\ns3grep --pattern \"ERROR\" --bucket my-logs-bucket --prefix logs/ --concurrent-tasks 16\n```\n\n### CLI Options\n\n| Flag                | Description                                 |\n|---------------------|---------------------------------------------|\n| `-p`, `--pattern`   | Search pattern (required)                   |\n| `-b`, `--bucket`    | S3 bucket name (required)                   |\n| `-z`, `--prefix`    | S3 prefix to search in (default: \"\")        |\n| `-c`, `--concurrent-tasks` | Number of concurrent tasks (default: 8) |\n| `-i`, `--case-sensitive`   | Case sensitive search                 |\n| `-q`, `--quiet`     | Hide progress bar                           |\n| `-n`, `--line-number` | Show line numbers in output               |\n\n---\n\n## Example\n\n```sh\ns3grep --pattern \"timeout\" --bucket my-bucket --prefix logs/2025/06/ --concurrent-tasks 12 --line-number\n```\n\n---\n\n## Testing\n\nIntegration tests use [Localstack](https://github.com/localstack/localstack) to mock S3. See [CONTRIBUTING.md](CONTRIBUTING.md) for details.\n\n---\n\n## Contributing\n\nContributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n---\n\n## License\n\nThis project is licensed under the [MIT License](LICENSE).\n\n---\n\n## Acknowledgments\n\n- Inspired by daily use of `grep` and the need for cloud-native log search.\n- Built with [Rust](https://www.rust-lang.org/), [aws-sdk-rust](https://github.com/awslabs/aws-sdk-rust), and [Localstack](https://github.com/localstack/localstack) for testing.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdacort%2Fs3grep","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdacort%2Fs3grep","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdacort%2Fs3grep/lists"}