{"id":13799033,"url":"https://github.com/clarketm/s3recon","last_synced_at":"2025-04-07T18:13:10.130Z","repository":{"id":49756346,"uuid":"169017216","full_name":"clarketm/s3recon","owner":"clarketm","description":"Amazon S3 bucket finder and crawler.","archived":false,"fork":false,"pushed_at":"2021-12-27T15:38:59.000Z","size":76,"stargazers_count":152,"open_issues_count":1,"forks_count":55,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-31T17:17:04.175Z","etag":null,"topics":["crawler","finder","python","recon","s3","s3-bucket"],"latest_commit_sha":null,"homepage":"https://s3recon.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/clarketm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-02-04T02:13:04.000Z","updated_at":"2025-03-17T22:00:58.000Z","dependencies_parsed_at":"2022-09-07T07:50:51.358Z","dependency_job_id":null,"html_url":"https://github.com/clarketm/s3recon","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clarketm%2Fs3recon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clarketm%2Fs3recon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clarketm%2Fs3recon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clarketm%2Fs3recon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clarketm","download_url":"https://codeload.github.com/clarketm/s3recon/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247704571,"owners_count":20982298,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","finder","python","recon","s3","s3-bucket"],"created_at":"2024-08-04T00:00:58.056Z","updated_at":"2025-04-07T18:13:10.094Z","avatar_url":"https://github.com/clarketm.png","language":"Python","funding_links":[],"categories":["Bucket Enumeration Tools"],"sub_categories":[],"readme":"# [s3recon](https://s3recon.readthedocs.io/en/latest/)\n\n[![PyPi release](https://img.shields.io/pypi/v/s3recon.svg)](https://pypi.org/project/s3recon/)\n[![PyPi versions](https://img.shields.io/pypi/pyversions/s3recon.svg)](https://pypi.org/project/s3recon/)\n[![Downloads](https://pepy.tech/badge/s3recon)](https://pepy.tech/project/s3recon)\n[![Documentation Status](https://readthedocs.org/projects/s3recon/badge/?version=latest)](https://s3recon.readthedocs.io/en/latest/?badge=latest)\n\nAmazon S3 bucket finder and crawler.\n\n\u003cbr\u003e\n\u003ca href=\"https://github.com/clarketm/s3recon\"\u003e\n  \u003cp align=\"center\"\u003e\u003cimg width=\"40%\" src=\"https://raw.githubusercontent.com/clarketm/s3recon/master/recon.jpeg\" /\u003e\u003c/p\u003e\n\u003c/a\u003e\n\n[Check out the s3recon docs](https://s3recon.readthedocs.io/en/latest/)\n\n## Installation\n\u003e **NOTE**: s3recon requires python version **\u003e=3.6**\n\n```bash\n$ pip install s3recon\n```\n\n## Usage\n```text\n\nusage: s3recon [-h] [-o file] [-d] [-p] [-t seconds] [-v] [-c num] word_list [word_list ...]\n\npositional arguments:\n  word_list                      read words from one or more \u003cword-list\u003e files\n\noptional arguments:\n  -h, --help                     show this help message and exit\n  -o file, --output file         write output to \u003cfile\u003e\n  -d, --db                       write output to database\n  -p, --public                   only include 'public' buckets in the output\n  -t seconds, --timeout seconds  http request timeout in \u003cseconds\u003e (default: 30)\n  -v, --version                  show program's version number and exit\n  -c num, --concurrency num      maximum \u003cnum\u003e of concurrent requests (default: # of lcpus)\n  \n```\n\n## Example 1: Output to a json file\n\n#### 1. Download a word-list. \nThe [SecLists](https://github.com/clarketm/s3recon/edit/master/README.md) repository has a multitude of word-lists to choose from. For this example, let's download the sample word-list included in this repository.\n\n```bash\n$ curl -sSfL -o \"word-list.txt\" \"https://raw.githubusercontent.com/clarketm/s3recon/master/data/words.txt\"\n```\n\n#### 2. Run `s3recon`. \nExecute `s3recon` using the `word-list.txt` file and output the `public` S3 buckets to a json file named `results.json`.\n\n```bash\n$ s3recon \"word-list.txt\" -o \"results.json\" --public\n\n- PRIVATE https://s3.sa-east-1.amazonaws.com/test-lyft\n- PRIVATE https://s3.ap-south-1.amazonaws.com/test.amazon\n+ PUBLIC https://walmart-dev.s3.us-east-1.amazonaws.com\n- PRIVATE https://s3.ap-southeast-1.amazonaws.com/apple-prod\n- PRIVATE https://walmart.s3.ap-southeast-1.amazonaws.com\n...\n```\n\n#### 3. Inspect the results. \nCheck the `results.json` output file to view the S3 buckets you have discovered!\n\n```bash\n$ cat \"results.json\"\n```\n\n```json\n{\n    \"public\": {\n        \"total\": 12,\n        \"hits\": [\n            \"https://walmart-dev.s3.us-east-1.amazonaws.com\",\n            \"https://apple-production.s3.ap-southeast-1.amazonaws.com\",\n            ...\n        ]\n    }\n}\n```\n\n\u003e **Note:** to include `private` buckets in the results omit the `-p, --public` flag from the command.\n\n#### 4. Crawl the results.\nEnumerate the static files located in each bucket and record the findings.\n\u003e Coming soon!\n\n\n## Example 2: Output to a MongoDB database\n\n#### 1. Download a word-list. \nThe [SecLists](https://github.com/clarketm/s3recon/edit/master/README.md) repository has a multitude of word-lists to choose from. For this example, let's download the sample word-list included in this repository.\n\n```bash\n$ curl -sSfL -o \"word-list.txt\" \"https://raw.githubusercontent.com/clarketm/s3recon/master/data/words.txt\"\n```\n\n#### 2. Start an instance of MongoDB\n```text\n$ docker run --name \"mongo\" -p 27017:27017 -v \"mongodb_data:/data/db\" -v \"mongodb_config:/data/configdb\" -d mongo\n```\n\n#### 3. Run `s3recon`. \nExecute `s3recon` using the `word-list.txt` file and output to MongoDB instance.\n\n```bash\n$ s3recon \"word-list.txt\" --db\n\n- PRIVATE https://s3.sa-east-1.amazonaws.com/test-lyft\n- PRIVATE https://s3.ap-south-1.amazonaws.com/test.amazon\n+ PUBLIC https://walmart-dev.s3.us-east-1.amazonaws.com\n- PRIVATE https://s3.ap-southeast-1.amazonaws.com/apple-prod\n- PRIVATE https://walmart.s3.ap-southeast-1.amazonaws.com\n...\n```\n\n#### 3. Inspect the results. \nCheck the MongoDB database: `s3recon` collection: `hits` to view the S3 buckets you have discovered!\n\n```bash\n$ mongo \"s3recon\" --quiet --eval 'db.hits.find({}, {\"url\": 1, \"access\": 1, \"_id\": 0}).limit(5)'\n```\n\n```json\n{ \"url\" : \"https://s3.us-east-2.amazonaws.com/apple\", \"access\" : \"private\" }\n{ \"url\" : \"https://s3.us-west-1.amazonaws.com/microsoft-dev\", \"access\" : \"private\" }\n{ \"url\" : \"https://s3.us-west-1.amazonaws.com/dev-microsoft\", \"access\" : \"private\" }\n{ \"url\" : \"https://s3.us-east-2.amazonaws.com/amazon\", \"access\" : \"private\" }\n{ \"url\" : \"https://s3.us-east-1.amazonaws.com/dev-amazon\", \"access\" : \"private\" }\n```\n\n#### 4. Crawl the results.\nEnumerate the static files located in each bucket and record the findings.\n\u003e Coming soon!\n\n\n## FAQ\n#### Q: How do I configure this utility?\n#### A: \n`s3recon` can be configure using a yaml configuration file located in either the current working directory (e.g. `./s3recon.yml`) or your home diretory (e.g. `~/s3recon.yml`).\n\nThe following is the list of configurable values:\n```yaml\n# s3recon.yml\n\ndatabase: { host: \"0.0.0.0\", ... }\n  \nseparators: [\"-\", \"_\", \".\"]\n\nenvironments: [\"\", \"backup\", \"backups\", ...]\n\nregions: [\"ap-northeast-1\", \"ap-northeast-2\", ...]\n```\n\n\u003e To see the full list of configurable values (and their **defaults**) please refer to the [s3recon.yml](https://github.com/clarketm/s3recon/blob/master/s3recon/s3recon.yml) file in this repository. \n\n\n#### Q: How do I customize the AWS regions used in the recon?\n#### A: \nThe AWS *regions* can be altered by setting the `regions` array in your `s3recon.yml` configuration file. \n```yaml\n# s3recon.yml\n\nregions: [ \"us-west-2\", ...]\n```\n\n\n#### Q: How do I customize the environment values used in the recon?\n#### A: \nThe *environments* are modifiers permuted with each item of the *word-list* (and the *separator*) to construct the bucket value in request.\nThe value can be altered by setting the `environments` array in your `s3recon.yml` configuration file.\n\nFor example, to only search lines from the word-list *verbatim* (i.e. without modification) you can set this value to an empty array. \n```yaml\n# s3recon.yml\n\nenvironments: []\n```\n\n#### Q: How do I customize the MongoDB host and port?\n#### A: \nThe database *host* and *port* can be configured by altering the `database` map in your `s3recon.yml` configuration file.\n\nFor example, `host` and `port` can be set directly inside the `database` map\n```yaml\n# s3recon.yml\n\ndatabase: {\n  host: \"0.0.0.0\",\n  port: 27017\n}\n```\n\n#### Q: How do I use a database other than MongoDB?\n#### A: \nSorry, at the moment only MongoDB is supported.\n\n## Going Forward\n\n- [ ] Create `crawl` command to crawl public/private buckets found in `find` stage.\n- [ ] Separate out `find` and `crawl` as subcommands.\n- [x] Store discovered buckets in a NoSQL database.\n\n## Disclaimer\nThis tools is distributed for educational and security purposes. I take no responsibility and assume no liability for the manner in which this tool is used.\n\n## License\n\nMIT \u0026copy; [**Travis Clarke**](https://blog.travismclarke.com/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclarketm%2Fs3recon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclarketm%2Fs3recon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclarketm%2Fs3recon/lists"}