https://github.com/hortonworks/cloud-haunter
Cloud agnostic resource monitoring and janitor tool
https://github.com/hortonworks/cloud-haunter
aws azure cloud gcp golang janitor monitoring
Last synced: 24 days ago
JSON representation
Cloud agnostic resource monitoring and janitor tool
- Host: GitHub
- URL: https://github.com/hortonworks/cloud-haunter
- Owner: hortonworks
- License: apache-2.0
- Created: 2018-03-13T16:13:56.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2026-05-28T12:47:14.000Z (29 days ago)
- Last Synced: 2026-05-28T14:20:43.369Z (29 days ago)
- Topics: aws, azure, cloud, gcp, golang, janitor, monitoring
- Language: Go
- Homepage:
- Size: 11.7 MB
- Stars: 21
- Watchers: 35
- Forks: 12
- Open Issues: 8
-
Metadata Files:
- Readme: README.MD
- License: LICENSE
Awesome Lists containing this project
README
[](https://circleci.com/gh/hortonworks/cloud-haunter) [](https://goreportcard.com/report/github.com/hortonworks/cloud-haunter)
# Cloud Haunter
`Haunter is a pokémon ghost which will poison and kill your unidentified and orphan cloud resources`
The main goal of this project is to save costs in the cloud by monitoring and terminating resources.
Cloud Haunter (ch) supports the three major cloud providers.
`The project is under construction, use on your own risk!`
## Supported resources
---
| | AWS | AZURE | GCP |
|----------|------------------------------------------------------|-----------------|--------------------------------|
| Stack | Cloudformation stack, Native stack assembled by tags | Resource group | Native stack assembled by tags |
| Instance | EC2 instance | Virtual machine | Compute Engine instances |
| Disk | EC2 disk | - | Compute Engine disks |
| Access | IAM user | - | IAM service accounts |
| Database | RDS database | - | SQL instances |
| Alert | CloudWatch alarm | - | - |
| Storage | - | Storage account | - |
### Filters appliable to resources:
* long running
* without owner
* actually running
* already stopped
* old cloud credentials
* resource unused
### Actions appliable to resources:
* send notification
* log result
* print result in json format
* stop instances [AWS, AZURE, GCP]
* terminate instances [AWS, AZURE, GCP]
* terminate stacks [AWS, AZURE, GCP]
* terminate disks [AWS, GCP]
* terminate images [AWS, AZURE, GCP]
* cleanup storages [AWS, AZURE]
## Prerequisites
---
For the proper work, you have to use some custom tags/labels on your cloud resources.
#### AWS
* Owner tag: `owner`
* Ignore tag: `cloud-cost-reducer-ignore`
* Resource grouping label: `Cloudera-Environment-Resource-Name` (for native stacks)
#### Azure
* Owner tag: `owner`
* Ignore tag: `cloud-cost-reducer-ignore`
* Resource creation time, because Azure SDK doesn't response it: `cb-creation-timestamp`
#### Google
* Owner label: `owner`
* Ignore label: `cloud-cost-reducer-ignore`
* Resource grouping label: `cloudera-environment-resource-name`
* Resource description (optional): `Generated by CDP.`
There is an option to use custom tags/labels, but you have to recompile the project with a bunch of custom environment variables like `OWNER_LABEL`. Please check Makefile.
### Resource filtering
This tool works well if you use it from early days of your cloud account and all of your users are following the basic rules of tagging instances. On the other hand introducing it on an existing environment should be pain in the back.
There is an option to declare your include/exclude policies in a YAML file (please have look at utils/testdata/filterConfig.yml).
CH will include/exclude all the resources where the name, owner, or any of the tags are matching with the given configuration.
## Installation
---
```
curl -Ls https://github.com/hortonworks/cloud-haunter/releases/download/v0.5.41/cloud-haunter_0.5.41_$(uname)_x86_64.tgz | tar x
```
**NOTE**: You can find the latest version released under https://github.com/hortonworks/cloud-haunter/releases
## Usage
---
```
NAME:
Cloud Haunter
USAGE:
ch -o=operation -a=action [-f=filter1,filter2] [-c=cloud1,cloud2]
VERSION:
v0.5.7-snapshot
AUTHOR(S):
Hortonworks
OPERATIONS:
-o getAccess
-o getAlerts
-o getDatabases
-o getDisks
-o getImages
-o getInstances
-o getStacks
-o getStorages
-o readImages
FILTERS:
-f failed
-f longrunning
-f match
-f nomatch
-f oldaccess
-f ownerless
-f running
-f stopped
-f unused
ACTIONS:
-a cleanup
-a json
-a log
-a notification
-a stop
-a termination
CLOUDS:
-c AWS_GOV
-c AWS
-c AZURE
-c GCP
FILTER_CONFIG:
-fc=/location/of/filter/config.yml
DRY RUN:
-d
VERBOSE:
-v
DISABLE_IGNORE_LABEL:
-i
EXACT_MATCH_OWNERS:
-e
HELP:
-h
```
### Environment variables
#### AWS
* AWS_ACCESS_KEY_ID
* AWS_SECRET_ACCESS_KEY
#### Azure
* AZURE_SUBSCRIPTION_ID
* AZURE_TENANT_ID
* AZURE_CLIENT_ID
* AZURE_CLIENT_SECRET
#### Google
* GOOGLE_PROJECT_ID
* GOOGLE_APPLICATION_CREDENTIALS, location of service account JSON
#### HipChat
* HIPCHAT_TOKEN
* HIPCHAT_SERVER
* HIPCHAT_ROOM
#### Slack
* SLACK_WEBHOOK_URL
#### Long running
* RUNNING_PERIOD, default: 24h
#### Old access
* ACCESS_AVAILABLE_PERIOD, default: 2880h
#### Retention days for cleanup
* RETENTION_DAYS, default: 90
### Usage examples
Terminate AWS stacks with owners not exactly matching the known owners, even if they have an ignore label
```
# Set up cloud provider settings
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
# Run Cloud Haunter
ch -o getStacks -a termination -f nomatch -c aws -fc known-owners-filter-config.yml -i -e
```
Terminate Azure instances with specific owners if they are running longer than 6 hours
```
# Set up cloud provider settings
export AZURE_SUBSCRIPTION_ID=...
export AZURE_TENANT_ID=...
export AZURE_CLIENT_ID=...
export AZURE_CLIENT_SECRET=...
# Set up additional parameters
export RUNNING_PERIOD=6h
# Run Cloud Haunter
ch -o getInstances -a termination -f longrunning,match -c azure -fc owner-filter-config-v2.yml -e
```
**NOTE**: You can find example filter config files under _utils/testdata_
## Development
### Dependencies
* Golang 1.15
* Docker
* GNU Make
### Building the project
Make sure you have the needed dependencies ony your development machine, then simply use make for building the project:
```
make build
```
You may also need to format your changes before building to pass the pre-build checks, you can do this with `make format`
### Adding a new dependency
* Import the dependency in your go code
* Run `make mod-tidy`
### Releasing a new version
```
VERSION={{new-semantic-version}} make release
```