Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/keithnoguchi/do-in-action
DO with Terraform and Ansible
https://github.com/keithnoguchi/do-in-action
Last synced: about 1 month ago
JSON representation
DO with Terraform and Ansible
- Host: GitHub
- URL: https://github.com/keithnoguchi/do-in-action
- Owner: keithnoguchi
- Created: 2017-05-30T21:14:24.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-06-05T06:35:13.000Z (over 6 years ago)
- Last Synced: 2024-08-23T11:06:56.699Z (4 months ago)
- Language: HCL
- Homepage:
- Size: 167 KB
- Stars: 12
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-digitalocean - DigitalOcean in Action! - Example repo for using DigitalOcean with Terraform and Ansible. (Configuration management)
README
# DigitalOcean in Action!
[![asciicast]](https://asciinema.org/a/3SlAFCVTsV2SYffiCiCgLiK5p)
Forming [DigitalOcean] [droplets] through [terraform].
[DigitalOcean]: http://digitalocean.com
[droplets]: http://www.digitalocean.com/products/compute/
[terraform]: http://terraform.io
[asciicast]: https://asciinema.org/a/3SlAFCVTsV2SYffiCiCgLiK5p.png
- [Setup](#setup)
- [Run](#run)
- [Test](#test)
- [Import](#import)
- [Benchmark](#benchmark)
- [Cleanup](#cleanup)
- [References](#references)
Install [terraform](http://terraform.io) and [ansible](http://ansible.com)
on your favorite laptop/desktop and enjoy having DO in action!
Note that, since [DO Cloud Firewalls](https://do.co/cloud-firewalls) support
has been just
[merged into master](https://github.com/terraform-providers/terraform-provider-digitalocean/pull/1)
recently, you either grab the latest DO provider release,
[0.1.0](https://github.com/terraform-providers/terraform-provider-digitalocean/blob/master/CHANGELOG.md#010-june-19-2017),
or git clone by yourself, if you're adventurous.
Here is a part of
[ansible playbook](https://github.com/keinohguchi/arch-on-air/blob/master/guest.yml#L33-L48)
I wrote for [arch-on-air](https://github.com/keinohguchi/arch-on-air)
to `git clone` and install the latest [terraform](http://terraform.io)
and [ansible](http://ansible.com). As both are under really active development,
use it on your own risk with hacker's mind. :)
## Setup
Sign in or sign up to [DigitalOcean](http://digitalocean.com) through
[cloud.digitalocean.com](https://cloud.digitalocean.com/registrations/new)
and grab a [DO APIv2 token](http://www.digitalocean.com/community/tutorials/how-to-use-the-digitalocean-api-v2).
And then, set the API token for the [terraform](http://terraform.io)
by defining `TA_VAR_DO_API_TOKEN` environment variable as below:
```sh
air$ export TF_VAR_DO_API_TOKEN=$(cat ~/.do/token.pem)
```
also, setup the SSH key in the setup section of the
[cloud.digitalocean.com](http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=12&ved=0ahUKEwjy1Z-Gs6LUAhWP2YMKHdz3ChUQFghUMAs&url=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-set-up-ssh-keys--2&usg=AFQjCNGf2nqGqjoCm0fCqIV3mR-djWG8qA)
and give it to [terraform](http://terraform.io) through
`TA_VAR_DO_FINGERPRINT` environment variable, so that you can
ssh into those droplets:
```sh
air$ export TF_VAR_DO_FINGERPRINT=$(ssh-keygen -E md5 -lf ~/.ssh/id_rsa.pub|awk '{print $2}'|sed 's/MD5://')
```
That's it! Now, let's roll!
## Run
As mentioned before, all the droplets are formed through
the [terraform](http://terraform.io), as you can see it
in [main.tf](main.tf) file. To make the operation
straightforward, however, I've wrote a simple [Makefile](Makefile).
- [Plan](#plan)
- [Deploy](#deploy)
### Plan
[terraform](http://terraform.io) has a way to dry run the
actual actions through `terraform plan`. There is a one-to-one
[Makefile](Makefile) target, called, `plan`:
```sh
air$ make plan
```
### Deploy
[Makefile](Makefile) deploy target is, as you can guess,
just a wrapper to the `terraform apply`:
```sh
air$ make deploy
```
this will droplets in the cloud. Through the use
of [terraform output variables](outputs.tf), you can check the
IP reachability to one of the droplets as below:
IPv4:
```sh
air$ ping -4 -c3 $(terraform output server0_public_ipv4)
PING 104.131.96.15 (104.131.96.15) 56(84) bytes of data.
64 bytes from 104.131.96.15: icmp_seq=1 ttl=56 time=81.9 ms
64 bytes from 104.131.96.15: icmp_seq=2 ttl=56 time=81.2 ms
64 bytes from 104.131.96.15: icmp_seq=3 ttl=56 time=78.3 ms
--- 104.131.96.15 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 78.335/80.504/81.943/1.578 ms
air$
```
IPv6:
```sh
air$ ping -6 -c3 $(terraform output server0_public_ipv6)
PING 2604:a880:0800:0010:0000:0000:319e:1001(2604:a880:800:10::319e:1001) 56 data bytes
64 bytes from 2604:a880:800:10::319e:1001: icmp_seq=1 ttl=49 time=81.4 ms
64 bytes from 2604:a880:800:10::319e:1001: icmp_seq=2 ttl=49 time=81.4 ms
64 bytes from 2604:a880:800:10::319e:1001: icmp_seq=3 ttl=49 time=79.5 ms
--- 2604:a880:0800:0010:0000:0000:319e:1001 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 79.521/80.806/81.467/0.908 ms
air$
```
and to the floating IP:
```sh
air$ ping -c3 $(cd flips && terraform output server_flip)
PING 45.55.97.179 (45.55.97.179) 56(84) bytes of data.
64 bytes from 45.55.97.179: icmp_seq=1 ttl=56 time=99.2 ms
64 bytes from 45.55.97.179: icmp_seq=2 ttl=56 time=97.6 ms
64 bytes from 45.55.97.179: icmp_seq=3 ttl=56 time=98.5 ms
--- 45.55.97.179 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 97.690/98.508/99.278/0.649 ms
air$
```
And, as we're running the simple HTTP server through
[the droplet user data](http://www.digitalocean.com/community/tutorials/an-introduction-to-droplet-metadata), you can check the HTTP
reachability to the server, as below:
```sh
air$ curl http://$(terraform output server0_public_ipv4)
Hello world from server0
air$
```
and through the floating IP:
```sh
air$ curl http://$(terraform output server_flip)
Hello world from server0
air$
```
## Test
Of course, you need a test to check those droplets. I use
[ansible](http://ansible.com) to drive all those tests by
[dynamically](http://docs.ansible.com/ansible/intro_dynamic_inventory.html)
retrieving the variables, IP addresses, etc., through the
[inventory.py](inventory.py). All you need to do is just
call `make test`, which deploy droplets, if it's not there,
and run all the ansible test playbooks:
```sh
air$ make test | tail -4
PLAY RECAP *********************************************************************
45.55.207.205 : ok=32 changed=0 unreachable=0 failed=0
air$
```
### Test matrix
Here is the test matrix covered by `make test-all`:
| Firewall type | [Non firewall](main.tf) | [Inbound deny-all](firewalls/server/000-deny-all/main.tf) | [Inbound allow-all](firewalls/server/999-allow-all/main.tf) | [Outbound deny-all](firewalls/client/000-deny-all/main.tf) | [Outbound allow-all](firewalls/client/999-allow-all/main.tf) | [In/Out-bound deny-all](firewalls/both/000-deny-all/main.tf) | [In/Out-bound allow-all](firewalls/both/999-allow-all/main.tf) |
|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| ICMP | [PASS](tests/icmp.yml) | [PASS](firewalls/server/000-deny-all/tests/icmp.yml) | [PASS](firewalls/server/999-allow-all/tests/icmp.yml) | [PASS](firewalls/client/000-deny-all/tests/icmp.yml) | [PASS](firewalls/client/999-allow-all/tests/icmp.yml) | [PASS](firewalls/both/000-deny-all/tests/icmp.yml) | [PASS](firewalls/both/999-allow-all/tests/icmp.yml) |
| ICMP port unreach | [PASS](tests/icmp_port_unreach.yml) | [PASS](firewalls/server/000-deny-all/tests/icmp_port_unreach.yml) | [PASS](firewalls/server/999-allow-all/tests/icmp_port_unreach.yml) | [PASS](firewalls/client/000-deny-all/tests/icmp_port_unreach.yml) | [PASS](firewalls/client/999-allow-all/tests/icmp_port_unreach.yml) | [PASS](firewalls/both/000-deny-all/tests/icmp_port_unreach.yml) | [PASS](firewalls/both/999-allow-all/tests/icmp_port_unreach.yml) |
| HTTP | [PASS](tests/http.yml) | [PASS](firewalls/server/000-deny-all/tests/http.yml) | [PASS](firewalls/server/999-allow-all/tests/http.yml) | [PASS](firewalls/client/000-deny-all/tests/http.yml) | [PASS](firewalls/client/999-allow-all/tests/http.yml) | [PASS](firewalls/both/000-deny-all/tests/http.yml) | [PASS](firewalls/both/999-allow-all/tests/http.yml) |
| TCP | [PASS](tests/tcp.yml) | [PASS](firewalls/server/000-deny-all/tests/tcp.yml) | [PASS](firewalls/server/999-allow-all/tests/tcp.yml) | [PASS](firewalls/client/000-deny-all/tests/tcp.yml) | [PASS](firewalls/client/999-allow-all/tests/tcp.yml) | [PASS](firewalls/both/000-deny-all/tests/tcp.yml) | [PASS](firewalls/both/999-allow-all/tests/tcp.yml) |
| UDP | [PASS](tests/udp.yml) | [PASS](firewalls/server/000-deny-all/tests/udp.yml) | [PASS](firewalls/server/999-allow-all/tests/udp.yml) | [PASS](firewalls/client/000-deny-all/tests/udp.yml) | [PASS](firewalls/client/999-allow-all/tests/udp.yml) | [PASS](firewalls/both/000-deny-all/tests/udp.yml) | [PASS](firewalls/both/999-allow-all/tests/udp.yml) |
| FTP | [PASS](tests/ftp.yml) | [PASS](firewalls/server/000-deny-all/tests/ftp.yml) | [PASS](firewalls/server/999-allow-all/tests/ftp.yml) | [PASS](firewalls/client/000-deny-all/tests/ftp.yml) | [PASS](firewalls/client/999-allow-all/tests/ftp.yml) | [PASS](firewalls/both/000-deny-all/tests/ftp.yml) | [PASS](firewalls/both/999-allow-all/tests/ftp.yml) |
Note that all those Ansible test playbooks covers four different
IP reachability, public IPv4, private IPv4, public IPv6, and
floating IPv4, respectively.
And also, you can run those tests individually through the standard
ansible command. For example, following command runs HTTP test for
non firewalls setup.
```sh
air$ ansible-playbook tests/http.yml
```
### Test against the entire DO
Here is the simple shell script to run the test against the entire DO!
```sh
air$ for region in ams2 ams3 blr1 fra1 lon1 nyc1 nyc2 nyc3 sfo1 sfo2 sgp1 tor1
do
# Cleanup the environment before the test.
if ! make clean-all test-all TF_VAR_DO_REGION=$region
then
echo 'oh, no!'
break
fi
done
```
## Import
Terraform has a feature called import, which imports the existing infrastructure,
e.g. droplets, by `terraform import` command. In this section, I'll demonstrate
how to use `import` sub command to import the existing droplets and apply
firewalls rules against it through the terraform.
Please note that the imported droplets and the one described in `main.tf` should
share the same attributes, e.g. droplet images, or terraform will destroy the one
you import and re-create new one, as it treats that as a conflict of the resources.
### Client
Let's import the existing droplet as a client role. For this, I use [doctl] for the
pre-creation but you can use any cloud orchestration tool of your choice.
```bash
air$ doctl compute droplet create client0 --image ubuntu-16-04-x64 --region nyc3 \
--size 512mb --enable-ipv6 --enable-private-networking \
--ssh-keys $TF_VAR_DO_FINGERPRINT --user-data-file ./scripts/client_user_data.sh
```
I'm using [./scripts/client_user_data.sh] as the user data, for `python` and `nmap`.
Once it's up and running, you can import the droplet with `terraform import`
command, as below. Only argument you need is the droplet ID and there are
multiple way to get that. For `doctl` case, it will show in the console, or
`doctl compute droplet list`.
```bash
air$ terraform import digitalocean_droplet.client 12345
```
Once it's successfully imported, all you have to do is just type `make` to let
`terraform` to do the rest, with one caveat.
As `terraform` will check the remote infrastructure against your local intention
to make sure it's in sync, it will compare all the attributes to make a decision.
But there is one field that force terraform to re-create droplets all the time,
which is the user data. This is because [DigitalOcean's APIv2] doesn't expose
the user data, and that makes terraform to always to destroy and re-create
to make your intention happens in the cloud. Since this is not what I want,
I've defined the environmental variable, called `TF_VAR_DO_CLIENT_USER_DATA`
and pass it nil, as below, to fool `terraform`.
```bash
air$ TF_VAR_DO_CLIENT_USER_DATA= terraform apply
```
To run the full test suite, you can do the same by replace it to `make test-all`,
as below:
```bash
air$ TF_VAR_DO_CLIENT_USER_DATA= make test-all
```
### Server
It's almost identical to use the existing droplet as a server role.
First create the one with `doctl`:
```bash
air$ doctl compute droplet create server0 --image ubuntu-16-04-x64 --region nyc3 \
--size 512mb --enable-ipv6 --enable-private-networking \
--ssh-keys $TF_VAR_DO_FINGERPRINT --user-data-file ./scripts/server_user_data.sh
```
then, import it as the server:
```bash
air$ terraform import digitalocean_droplet.server 54321
```
now, run it with `TF_VAR_DO_SERVER_USER_DATA=`:
```bash
air$ TF_VAR_DO_SERVER_USER_DATA= terraform apply
```
To run the full test suite, you can do the same by replace it to `make test-all`,
as below:
```bash
air$ TF_VAR_DO_SERVER_USER_DATA= make test-all
```
[DigitalOcean's APIv2]: https://developers.digitalocean.com/documentation/v2/#retrieve-an-existing-droplet-by-id
## Benchmark
You can run the [Cloud Bandwidth Performance Monitoring] based benchmark
against the server side droplet. Currently, it executes both the upload
and the download bandwidth benchmark with `iperf3`. The duration of
the single benchmark is 10 seconds each. It runs for an hour with 40
seconds interval to avoid the excessive load on the cloud.
You can execute for each IP address type, public and private IPv4, and
IPv6, with the simple make target.
Here is the public IPv4 benchmark:
```bash
air$ make bench-ipv4
```
and the private IPv4 equivalent:
```bash
air$ make bench-ipv4-private
```
Those make target will kick the ansible playbook, [ipv4.yml] and
[ipv4_private.yml] respectively, which first spin up the data vis
containers on the client droplet. This takes a while, as it fetches docker
images, and run it fresh. Once that ansible task is complete, you can point
your browser to the graphana dashboard, for example:
```bash
air$ chromium $(terraform output monitor_public_ipv4):8000
```
The playbook keep continue to run the `iperf3` server on the server droplet
and run the `iperf3` based bandwidth polling container for an hour. You can
monitor the trend through the graphana dashboard as mentioned above.
[ipv4.yml]: ./bench/ipv4.yml
[ipv4_private.yml]: ./bench/ipv4_private.yml
## Cleanup
Of course, we can destroy all those instances through `make clean`,
which is just calling `terraform destroy` in addition to cleaning
up the terraform state files:
```sh
air$ make clean
```
## References
Many thanks to [Yevgeniy (Jim) Brikman] and [Mitchell Anicas] for
their [terraform] insights! And [Brent Salisbury] for his cloud
monitoring demonstrated in his [Cloud Bandwidth Performance Monitoring]
github repository.
- [Terraform Up & Running]
- [How to use Terraform with DO]
- [Cloud Bandwidth Performance Monitoring] with Docker
[Yevgeniy (Jim) Brikman]: http://www.ybrikman.com/
[Mitchell Anicas]: http://twitter.com/thisismitch
[Brent Salisbury]: http://twitter.com/networkstatic
[Terraform Up & Running]: http://shop.oreilly.com/product/0636920061939.do
[How to use Terraform with DO]: http://www.digitalocean.com/community/tutorials/how-to-use-terraform-with-digitalocean
[Cloud Bandwidth Performance Monitoring]: https://github.com/nerdalert/cloud-bandwidth
### ASCII casts
I've recorded the [asciinema](https://asciinema.org) cast for the droplet creation part.
I'll upload it to the official site soon, but in a meanwhile, you can watch it with:
```sh
$ git clone https://github.com/keinohguchi/do-in-action
$ cd do-in-action
$ asciinema play ./casts/do-in-action.json
```
Happy Hacking!