https://github.com/jrhouston/grenade

Cause fires in production
https://github.com/jrhouston/grenade

chaos-engineering devops operations production-engineering

Last synced: 3 months ago
JSON representation

Cause fires in production

• Host: GitHub
• URL: https://github.com/jrhouston/grenade
• Owner: jrhouston
• Created: 2019-03-17T23:02:12.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2020-02-06T05:09:42.000Z (almost 5 years ago)
• Last Synced: 2023-03-04T22:36:11.396Z (almost 2 years ago)
• Topics: chaos-engineering, devops, operations, production-engineering
• Language: Go
• Homepage:
• Size: 8.46 MB
• Stars: 15
• Watchers: 2
• Forks: 0
• Open Issues: 16
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Grenade 💣 🔥 [![Go Report Card](https://goreportcard.com/badge/github.com/jrhouston/grenade)](https://goreportcard.com/report/github.com/jrhouston/grenade)

![](https://media.giphy.com/media/9PkfGzhKwBDHPTnDSj/giphy.gif)

`grenade` is a tool for _causing_ fires in production. I created this tool at [Wattpad](http://wattpad.com) to answer questions like:

- What happens when a developer deploys a change that chews up a ton of memory?

- How quickly do I get an alert when log volume in production spikes dramatically?

- Can a misbehaving service interfere with its neighbors in my production environment?

This is a useful activity to do particularly when operating at a scale where the blast radius of a failure in production can be quite large. For example, if you run a system where you have services with 300 replicas, it's good to know what's going to happen when you toss 300 grenades into production before someone inevitably does it by accident. This can help you:

- Test out your observability tools.

- Figure out what checks to put in place before a service is allowed to completely roll out in production.

- Carry out fire drills to train on-call people.

- Tune your monitors for pager alerts.

- Highlight gaps in monitoring and resource limits.

- See how resilient your infrastructure really is to unexpected failures.

⚠️ **DISCLAIMER** ⚠️

---

`grenade` does not _simulate_ problems by faking monitoring data. It actually causes problems. Only deploy this to testing and pre-production environments, or if you want to upset your on-call engineer.

## Build

```make build``` will produce a binary called `grenade` at the top level.

## Usage

`grenade` is a simple command line utility, see `grenade --help`

```

usage: grenade []

Flags:

  --help                   Show context-sensitive help (also try --help-long and --help-man).

  --port=8080              Webserver port

  --delay=DELAY            trigger the grenade after specified delay e.g 5m30s

  --crash                  Process crash

  --exit                   Exit process with non-zero exit code

  --mem-spike              Cause a memory spike

  --mem-leak               Cause a memory leak

  --cpu-spike              Cause a CPU spike

  --goroutine-leak         Cause goroutines to be leaked

  --conn-spike=CONN-SPIKE  Cause a connect spike to an address

  --conn-leak=CONN-LEAK    Cause a connect leak to an address

  --dns-spike              Cause a spike in DNS lookups

  --log-spike              Cause a spike in log volume

  --fd-spike               Cause a spike in open file descriptors

  --disk-spike             Cause a spike in disk space usage

  --disk-leak              Cause a leak in disk space usage

```