{"id":16567202,"url":"https://github.com/zloeber/aws-aware","last_synced_at":"2025-07-17T02:40:22.883Z","repository":{"id":42649552,"uuid":"240597071","full_name":"zloeber/aws-aware","owner":"zloeber","description":"AWS instance monitoring script for teams","archived":false,"fork":false,"pushed_at":"2023-02-07T23:40:38.000Z","size":224,"stargazers_count":0,"open_issues_count":7,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-15T23:43:13.769Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zloeber.png","metadata":{"files":{"readme":"README.MD","changelog":"HISTORY.rst","contributing":"CONTRIBUTING.rst","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.rst","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-14T20:55:29.000Z","updated_at":"2020-06-02T21:39:01.000Z","dependencies_parsed_at":"2024-11-15T11:28:50.161Z","dependency_job_id":null,"html_url":"https://github.com/zloeber/aws-aware","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zloeber%2Faws-aware","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zloeber%2Faws-aware/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zloeber%2Faws-aware/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zloeber%2Faws-aware/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zloeber","download_url":"https://codeload.github.com/zloeber/aws-aware/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242008883,"owners_count":20056983,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T21:05:56.315Z","updated_at":"2025-03-05T10:28:30.407Z","avatar_url":"https://github.com/zloeber.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## AWS-Aware - AWS Instance Threshold Monitor\n\nAWS Instance threshold monitoring bot. \n\n## Description\n\nThis app manually polls AWS for ec2 instance information based on tags, saves all data into a local file (cache), then uses that cached data to generate per team/environment instance threshold notifications that emit nice html reports based on tag filters.\n\n## Requirements\n\nI ran into issues installing dependencies (specifically psutil) without installing the following system library:\n\n```bash\nsudo apt install python3-dev\n```\n\n## Install\n\nAll install options assume you will be targeting a user or virtual python environment because that's the wise way to manage your local packages right?\n\n**Install For Development**\n\nClone this repo then jump into it:\n\n```bash\ngit clone ssh://git@github.com/zloeber/aws-aware\ncd aws-aware\n```\n\nInstall locally so that the installed binary is simply a shell for calling the files within the current folder. This allows for you to run the cli interactively while developing this app:\n\n```bash\npip install -f requirements.txt --user\npip install -e . --user\n```\n\nOr on a Linux/Mac with Python and pip installed:\n\n```bash\nmake install-deps install-local\n```\n\n**Install Locally (Preferred)**\n\nInstall from some custom artifactory via pip to the user workspace:\n\n```bash\npip install aws-aware --user --extra-index-url https://pypi.contoso.com/artifactory/api/pypi/internal-pypi-server/simple\n```\n\nYou can create your own virtual environment to install this module as well using pipenv or virtualenvwrapper.\n\n```bash\npip install pipenv --user\ncd aws-aware\npipenv install\npipenv shell\n```\n\nCalling the script in a virtual environment may prevent the script from being able to be run concurrently though and should not be used in production (unless possibly in a virtualenv within a container).\n\n## Initialization\n\nEasy! Copy `./config/deploy.env` to `./config/team1.env` then initialize a new aws-aware configuration file:\n\n```bash\nmake app/config/init dpl=./config/team1.env\n```\nAfter running this you will end up with a configuration file that you defined in `./config/team1-config.yml` as well as a `./config/team1-monitor.yml` config. Both can then be used moving forward to generate alerts for team1 (these config files are also able to be upgraded if the app upgrades for any reason).\n\nYou can then run other make tasks using the same configuration file:\n\n```bash\n# Run an instance debug export based on the previously created config file\nmake app/config/show dpl=team1.env\n```\n\n\u003e **NOTE:** aws-aware supports exporting configuration to self-updating configuration as code! `make app/config/export dpl=team1.env`\n\n## Upgrade\n\nUpgrading is similar to the initial install:\n\n```bash\npip install aws-aware --user --extra-index-url https://pypi.contoso.com/artifactory/api/pypi/internal-pypi-server/simple -U\n```\n\nOr to force re-install:\n\n```bash\npip install aws-aware --user --extra-index-url https://pypi.contoso.com/artifactory/api/pypi/internal-pypi-server/simple -U --no-cache-dir -I\n```\n\nIf you are upgrading the package then it is important to run the config file upgrade operation against any of your config files to validate if any new, pertinant, configuration entries have been added between versions. Upgrade with the following:\n\n```bash\naws-aware config upgradeconfig\n\n# Or against your custom configuration file\naws-aware -configfile awsaware-globalconfig.yml config upgradeconfig\n```\n\n## Usage\n\nThe script uses the Python click module for parsing CLI arguments. This makes it very similar to the tower-cli command line tool in usage.\n\n\u003e **NOTE:** If running this script under bash (via WSL, or on Linux/Mac) you can enable command line autocompletion with the following command in your bash profile:\n\u003e `eval \"$(_AWS_AWARE_COMPLETE=source aws-aware)\"`\n\nAs such it is critical to be aware that using the context sensitive help within the cli itself is preferred to this documentation as it will always contain the most recent features and flags. All remaining documentation is for general reference and getting started.\n\n## Configuration\n\nThe first time aws-aware is run, it will create a global configuration file to use automatically within its own install location (usually $HOME/.local/bin). You can (and should) use an alternate script configuration file by passing it first as `-configfile` so that the global configuration is not bound to the default install path pip uses for the module. You can create a new default configuration file to use via the new sub-command.\n\nYou can view your current configuration in any editor or via this command:\n```bash\naws-aware config show\n\n# Or against your custom configuration file\naws-aware -configfile awsaware-globalconfig.yml config show\n```\n\n\u003e **IMPORTANT** If this is your first time installing the application it would be wise to create a new global configuration file and working job directory within a secured location then commit the file to your repo (Config as code). This is almost mandatory if the app is installed and used in a shared environment.\n\n## Monitors\n\nBy default, no instance monitoring is performed. In order to monitor on instance count you will need to define them declaratively in a monitoring config yaml file. Below is an example of `config/prodenv-monitors.yml` that one might use to monitor production instances of types 'i2.xlarge' and 'r3.4xlarge'.\n\n```yaml\n# Monitor Configuration File for aws-aware\nview:\n  instance_tags: ['ClusterName', 'Environment', 'Location', 'Name', 'OS', 'PatchDay', 'Purpose', 'Stack']\n  instance_state: ['running']\n  notice_columns: ['ClusterName', 'Environment', 'Name', 'Stack']\n  # Tags on the left get transposed to column headers on the right\n  column_lookup:\n    ClusterName: 'Cluster Name'\n    instance_type: 'Instance Type'\n    Environment: 'Env'\n    launch_time: 'Launched'\n    public_ip_address: 'Public IP'\n    private_ip_address: 'Private IP'\n    name: 'Name'\nfilters:\n  Environment: 'Production'\nmonitors:\n  - name: i2.xlarge\n    thresholdtype: instance\n    warningthreshold: 0\n    alertthreshold: 1\n    enabled: true\n  - name: r3.4xlarge\n    thresholdtype: instance\n    warningthreshold: 0\n    alertthreshold: 1\n    enabled: true\n```\n## Uninstalling\n\n```bash\npip uninstall aws-aware\n```\n\n## Logging\n\nLogging is not enabled by default. If you need additional insight on what is going on for a job you will need to enable logging in the config/config.yml file. Relevant configuration settings are:\n\n```\nloggingenabled: false\nloglevel: \"INFO\"\nlogpath: \"logs/\"\nlogfile: \"runtime.log\"\nlogformat: \"%(relativeCreated)6d %(threadName)s %(message)s\"\n```\n\nSimply set loggingenabled to true and run the script again to get log output in the logs/runtime.log file (for long running jobs `tail -f logs/runtime.log`). This will emit logs for AWS and other calls.\n\n## Email Notifications\n\nIf warningnotice or alertnotice flags are sent to aws-aware and either thresholds have been reached for any monitor then an alert will be sent to any defined emailrecipients.\n\n### Slack Notifications\n\n**NOTE:** This is barely tested at all.\n\nNotifications can alternatively be sent to a Slack channel via a webhook. You can add a webhook for your channel [at the slack api site](https://get.slack.help/hc/en-us/articles/115005265063-Incoming-WebHooks-for-Slack#set-up-incoming-webhooks)\n\nYou can then add one or multiple hooks into your global configuration file in the slack_webhooks array. Then change slack_notifications to True. Once this has been done anytime a notification is triggered so too will each of the slack channels receive notifications.\n\n### AWS SSO\nTo use SSO based authentication you will need to ensure your existing credential file exists and has a dedicated profile to use. In this case, we will use the profile called 'saml' to maintain our session token information once authenticated via ADFS/SSO.\n\n```bash\nmkdir -p ${HOME}/.aws/credentials\necho '[saml]' \u003e\u003e ${HOME}/.aws/credentials\n```\nThen clone the following project or use something similar to it for saml authentication updates to your local aws credentials file:\n\n```bash\nexport ARG_USERNAME=consultant@contoso.com\nexport ARG_USERPASS='Lan ID Password'\nmake sso-login\n```\n\nThis will call a python script that will use the passed env vars to attempt to perform sso authentication then update the local saml profile with token/session information within the profile. You can use the script within (aws-sso.py) stand-alone to update a local aws profile for SAML authentication (tested as working on Windows/Mac/Linux with Python 2.7 only).\n\n\u003e**NOTE** Be careful to only specify dedicated profile names when using this script or you risk overwriting local ~/.aws/credentials information that other projects may depend upon. Default profile it will target is `saml`.\n\n## CI Steps\nSome common CI steps for this project includes:\n\n```bash\n# Code coverage reports\nmake coverage\n\n# Code linting reports\nmake lint\n\n# Unit test reports\nmake tests\n\n# Test against all versions of python via tox\nmake test-all\n\n# Update documentation\nmake docs\n```\nMore information on developing this project be found in the contribution document. \n\n## CD Steps\n\nThis is a quick list of steps to fully update and publish this project to Artifactory.\n\nTo start you will need to modify your `${HOME}\\.pypirc` profile to look similar to the following:\n\n```bash\n[distutils]\nindex-servers = internal-pypi-server\n[internal-pypi-server]\nrepository: https://pypi.contoso.com/artifactory/api/pypi/internal-pypi-server\nusername: \u003cid\u003e\npassword: \u003cpassword\u003e\n```\nYou will want to run this from an os with the make utility (alternatively pull out the commands from the Makefile and manually run them)\n\n1. First update HISTORY.rst with notes on your updates.\n2. Then run the following if you are releasing a new release:\n\n```bash\nmake docs\ngit add .\ngit commit -m '0.2.x Release'\nbumpversion patch\ngit push origin master\ngit push --tags\nmake release\n```\n\nOr, run the following to update the current release:\n\n```bash\nmake docs\ngit add .\ngit commit -m '0.2.x Release'\ngit push origin master\nmake release\n```\n\n## Credits\n\nMore than this I'm sure but here is how the project was bootstraped:\n\n[Cookiecutter:](https://github.com/audreyr/cookiecutter)\n\n[elgertam/cookiecutter-pipenv:](https://github.com/elgertam/cookiecutter-pipenv)\n\n[audreyr/cookiecutter-pypackage:](https://github.com/audreyr/cookiecutter-pypackage)\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzloeber%2Faws-aware","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzloeber%2Faws-aware","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzloeber%2Faws-aware/lists"}