{"id":21893785,"url":"https://github.com/goodwaygroup/gw-aws-audit","last_synced_at":"2025-04-15T15:22:49.971Z","repository":{"id":37025316,"uuid":"258343833","full_name":"GoodwayGroup/gw-aws-audit","owner":"GoodwayGroup","description":"A set of commands to audit AWS usage to identify cost savings and security issues.","archived":false,"fork":false,"pushed_at":"2024-04-12T21:05:20.000Z","size":267,"stargazers_count":8,"open_issues_count":12,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-04-14T16:14:22.486Z","etag":null,"topics":["audit-aws","aws","aws-cli","aws-ec2","aws-s3","clearing","cost-optimization","go","golang","goodwaygroup","s3-buckets"],"latest_commit_sha":null,"homepage":"https://goodwaygroup.github.io/gw-aws-audit/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GoodwayGroup.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":"audit.sh","citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-04-23T22:17:59.000Z","updated_at":"2024-04-16T20:44:58.271Z","dependencies_parsed_at":"2023-10-17T04:52:21.819Z","dependency_job_id":"5a969352-87e6-4049-b299-f8b3e3a2ea71","html_url":"https://github.com/GoodwayGroup/gw-aws-audit","commit_stats":null,"previous_names":[],"tags_count":51,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoodwayGroup%2Fgw-aws-audit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoodwayGroup%2Fgw-aws-audit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoodwayGroup%2Fgw-aws-audit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoodwayGroup%2Fgw-aws-audit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GoodwayGroup","download_url":"https://codeload.github.com/GoodwayGroup/gw-aws-audit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249095404,"owners_count":21211915,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audit-aws","aws","aws-cli","aws-ec2","aws-s3","clearing","cost-optimization","go","golang","goodwaygroup","s3-buckets"],"created_at":"2024-11-28T13:16:55.079Z","updated_at":"2025-04-15T15:22:49.942Z","avatar_url":"https://github.com/GoodwayGroup.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GW AWS Audit Tool\n\u003e NOTE: This is a specialized tool to help with actions to take during an audit of AWS usage.\n\n[![Go Report Card](https://goreportcard.com/badge/GoodwayGroup/gw-aws-audit)](https://goreportcard.com/report/GoodwayGroup/gw-aws-audit)\n\n## Basic Usage\n\nPlease see [the docs for details on the commands.](./docs/gw-aws-audit.md)\n\nUseful for clearing **large S3 buckets (many millions of objects)**, identifying egress EBS volumes and tracking S3 spend.\n\n```\n$ gw-aws-audit help\nNAME:\n   gw-aws-audit - a collection of tools to audit AWS.\n\nUSAGE:\n   gw-aws-audit [global options] command [command options] [arguments...]\n\nAUTHOR:\n   Derek Smith \u003cdsmith@goodwaygroup.com\u003e\n\nCOMMANDS:\n   s3               S3 related commands\n   rds              RDS related commands\n   ec2              EC2 related commands\n   sg               Security Group related commands\n   iam              IAM related commands\n   cw               CloudWatch related commands\n   install-manpage  Generate and install man page\n   version, v       Print version info\n   help, h          Shows a list of commands or help for one command\n\nGLOBAL OPTIONS:\n   --help, -h  show help (default: false)\n\nCOPYRIGHT:\n   (c) 2020 Goodway Group\n```\n\n## Installation\n\n### [`asdf` plugin](https://github.com/GoodwayGroup/asdf-gw-aws-audit)\n\nAdd plugin:\n\n```\n$ asdf plugin-add gw-aws-audit https://github.com/GoodwayGroup/asdf-gw-aws-audit.git\n```\n\nInstall the latest version:\n\n```\n$ asdf install gw-aws-audit latest\n```\n\n### [Homebrew](https://brew.sh) (for macOS users)\n\n```\nbrew tap GoodwayGroup/gw-aws-audit\nbrew install gw-aws-audit\n```\n\n### curl binary\n\n```\n$ curl https://i.jpillora.com/GoodwayGroup/gw-aws-audit! | bash\n```\n\n### [docker](https://www.docker.com/)\nThe compiled docker images are maintained on [GitHub Container Registry (ghcr.io)](https://github.com/orgs/GoodwayGroup/packages/container/package/gw-aws-audit).\nWe maintain the following tags:\n\n- `edge`: Image that is build from the current `HEAD` of the main line branch.\n- `latest`: Image that is built from the [latest released version](https://github.com/GoodwayGroup/gw-aws-audit/releases)\n- `x.y.z` (versions): Images that are build from the tagged versions within Github.\n\n```bash\ndocker pull ghcr.io/goodwaygroup/gw-aws-audit\ndocker run -v \"$PWD\":/workdir ghcr.io/goodwaygroup/gw-aws-audit --version\n```\n\n### man page\n\nTo install `man` page:\n\n```\n$ gw-aws-audit install-manpage\n```\n\n## Audit helper script\n\nThere is a bash helper script in the repo at [audit.sh](audit.sh). This tool is useful in running an audit across many regions at once.\n\n```\n$ ./audit.sh\n\naudit.sh helper script for gw-aws-audit\n\nUsage:\n    audit.sh [gw-aws-audit commands]\n\nExamples:\n\u003e This will run the 'gw-aws-audit sg detached' command for every region in the US (default)\n\n    $ audit.sh sg detached\n\n\u003e This will run the 'gw-aws-audit ec2 stopped-hosts' for ONLY the us-west-2 region\n\n    $ AWS_REGION=us-west-2 audit.sh ec2 stopped-hosts\n\n\u003e This will run the 'gw-aws-audit ec2 stopped-hosts' for every region in the EU\n\n    $ REGION=eu audit.sh ec2 stopped-hosts\n\n\u003e This will run the 'gw-aws-audit cw monitoring' using a specific version of the tool.\n\n    $ BIN_PATH=./bin/gw-aws-audit audit.sh cw monitoring\n\nNote: REGION env values (default: US):\nUS: us-east-1 us-east-2 us-west-1 us-west-2\nEU: eu-central-1 eu-west-1 eu-west-2 eu-west-3 eu-south-1 eu-north-1\nAP: ap-east-1 ap-south-1 ap-northeast-3 ap-northeast-2 ap-southeast-1 ap-southeast-2 ap-northeast-1\nCH: cn-north-1 cn-northwest-1\nROW: af-south-1 me-south-1 sa-east-2\nALL: All of the above combined\n\nYou can also set AWS_REGION and that will supersede the value of REGION\n✔ Have fun!\n```\n\n### Command Categories\n\nThere are commands for `s3`, `ec2`, `rds`, `sg` and `cw`\n\n**s3**\n```\n$ gw-aws-audit s3\n...\nCOMMANDS:\n   add-cost-tag                Add s3-cost-name to all S3 buckets\n   metrics                     Get usage metrics\n   clear-bucket, exterminatus  Clear all Objects within a given Bucket\n```\n\n**ec2**\n```\n$ gw-aws-audit ec2\n...\nCOMMANDS:\n   enhanced-monitoring  Produce report of Enhanced Monitoring enabled instances\n   detached-volumes     List detached EBS volumes and snapshot counts\n   stopped-hosts        List stopped EC2 hosts and associated EBS volumes\n```\n\n**rds**\n```\n$ gw-aws-audit rds\n...\nCOMMANDS:\n   enhanced-monitoring  Produce report of Enhanced Monitoring enabled instances\n```\n\n**cw**\n```\n$ gw-aws-audit cw\n...\nCOMMANDS:\n   enhanced-monitoring  Produce report of Enhanced Monitoring enabled EC2 \u0026 RDS instances\n```\n\n### Example Outputs\n\n#### ec2 stopped-hosts\n```\n$ gw-aws-audit ec2 stopped-hosts\n                INSTANCE ID          NAME            VOLUME                 SIZE (GB)  SNAPSHOTS  MIN SIZE (GB)  COSTS\n                i-09e42474f22039e23  dummy-box-test\n                                                     vol-0d4b4a7bc95a4b8e4          8          0              0  $0.80\n                                                     vol-0cc0f6cd3c99bc1cc          8          0              0  $0.80\n                TOTALS               1 INSTANCES     2 VOLUMES                  16 GB          0           0 GB  $1.60\n```\n\n#### ec2 detached-volumes\n```\n$ gw-aws-audit ec2 detached-volumes\n           VOLUME                 SIZE (GB)  SNAPSHOTS  MIN SIZE (GB)  COSTS\n           vol-0cc0f6cd3c99bc1cc          8          0              0  $0.80\n   TOTALS  1 VOLUMES                   8 GB          0           0 GB  $0.80\n```\n\n#### cw enhanced-monitoring\n```\n$ gw-aws-audit cw enhanced-monitoring\nEnhanced Metrics can add a cost. See: https://aws.amazon.com/cloudwatch/pricing/\nChecking for EC2 Enhanced Monitoring\n\n NAME                                              INSTANCE ID\n master-us-east-1c.masters.us-east-1.gwdocker.com  i-041h4jk12jk23sd\n jumpbox12                                         i-02412sdfgsgdfgs\n analytics-prod                                    i-0a87d921n1rtasd\n EC2 INSTANCES                                     3\n\n\nChecking for RDS Enhanced Monitoring\n\n DB INSTANCE                                ENGINE\n airflow-womp-ba-prod-v2                    postgres\n dashboard-reporting-production-instance-1  aurora-postgresql\n service-loloyol-production                 aurora-mysql\n DB INSTANCES                               3\n```\n\n#### s3 clear-bucket\n```\n$ gw-aws-audit s3 clear-bucket --bucket yolo\n-- WARNING -- PAY ATTENTION -- FOR REALS --\nThis will delete ALL objects in yolo\n-- THIS ACTION IS NOT REVERSIBLE --\nAre you SUPER sure? [yolo]\nEnter a value: yolo\n\nProceeding with batch delete for bucket: yolo\nPages: 198788 Listed: 198788000 Deleted: 198781000 Retries: 18373 DPS: 1921.64\n```\n\n#### s3 metrics\n```\n$ gw-aws-audit s3 metrics \u003e out.csv\nStarting metrics pull...\nBucket metric pull complete. Buckets: 207 Processed: 207\n```\n\n## Built With\n\n* go v1.14+\n* make\n* [git-chglog](https://github.com/git-chglog/git-chglog)\n* [goreleaser](https://goreleaser.com/install/)\n\n## Deployment\n\nRun `./release.sh $VERSION`\n\nThis will update docs, changelog, add the tag, push main and the tag to the repo. The `goreleaser` action will publish the binaries to the Github Release.\n\nIf you want to simulate the `goreleaser` process, run the following command:\n\n```\n$ curl -sL https://git.io/goreleaser | bash -s -- --rm-dist --skip-publish --snapshot\n```\n\n## Contributing\n\nPlease read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct, and the process for submitting pull requests to us.\n\n1. Fork the [GoodwayGroup/gw-aws-audit](https://github.com/GoodwayGroup/gw-aws-audit) repo\n1. Use `go \u003e= 1.16`\n1. Branch \u0026 Code\n1. Run linters :broom: `golangci-lint run`\n    - The project uses [golangci-lint](https://golangci-lint.run/usage/install/#local-installation)\n1. Commit with a Conventional Commit\n1. Open a PR\n\n## Versioning\n\nWe employ [git-chglog](https://github.com/git-chglog/git-chglog) to manage the [CHANGELOG.md](CHANGELOG.md). For the versions available, see the [tags on this repository](https://github.com/GoodwayGroup/gw-aws-audit/tags).\n\n## Authors\n\n* **Derek Smith** - [@clok](https://github.com/clok)\n\nSee also the list of [contributors](https://github.com/GoodwayGroup/gwvault/contributors) who participated in this project.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details\n\n## Sponsors\n\n[![goodwaygroup][goodwaygroup]](https://goodwaygroup.com)\n\n[goodwaygroup]: https://s3.amazonaws.com/gw-crs-assets/goodwaygroup/logos/ggLogo_sm.png \"Goodway Group\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoodwaygroup%2Fgw-aws-audit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoodwaygroup%2Fgw-aws-audit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoodwaygroup%2Fgw-aws-audit/lists"}