Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/bitgn/fdb-cloud-test

Packer + Terraform setup to experiment with FDB clusters in the cloud.
https://github.com/bitgn/fdb-cloud-test

foundationdb packer terraform

Last synced: 3 months ago
JSON representation

Packer + Terraform setup to experiment with FDB clusters in the cloud.

Awesome Lists containing this project

README 

        
# About



The purpose of this project is to make it easy to run various load

tests on systems with FoundationDB in the cloud.



We can achieve that by:



1. "Prebaking" cloud-specific VM images for the FoundationDB cluster

   nodes along with the tester nodes and machines for capturing the

   telemetry.

2. Using these images to quickly create FoundationDB clusters with the

   specific configuration.



The first step is handled by the [Packer](https://www.packer.io), the

second - by the [Terraform](https://www.terraform.io).



We want to start with a fixed topology:



- foundationDB nodes in the same network, forming a cluster;

- tester machines connected to the FDB.



Number and VM type for the node and load tester machines could be

changed within the configuration.



# Creating AMIs



First you need to create packer images for FDB and tester

machines. 



Note, that in this setup we actually create a temporary EC2 instance

in the cloud, install all the dependencies there, make a snapshot

(AMI) and kill the original machine. With these snapshots we could

then quickly launch a dozen of EC2 instances, connecting them into a

cluster.



You can create a FoundationDB AMI like this (make sure to fill in your

AWS credentials):



```

$ export AWS_ACCESS_KEY=""

$ export AWS_SECRET_KEY=""

$ cd packer-fdb

$ packer build --only=aws packer.json

```



Tester AMI can be created exactly like this but you enter

`packer-tester` folder instead.



## Creating Docker



Should you want to create docker images instead of AMIs, you can do so

via:



```

$ cd packer-fdb

$ packer build --only=docker packer.json

```



# Deploying Clusters



## Setup Terraform



Before deploying pre-baked images into AWS you need to configure your

working copy first. Install Terraform, then go to the `terraform`

folder of this repository and execute:



```

$ terraform init

```



Then, you would need to create a file `.secret.aws.tfvars`, filling it

with your AWS credentials:



```

aws_access_key = ""

aws_secret_key = ""

aws_account_id = ""

```



Afterwards you would also need to create a new ssh key called

`terraform` and place it into your `~/.ssh/` folder. Terraform will

install it into all the new machines, making it possible for you to

connect to them via ssh.



You can do that with:



```

$ ssh-keygen -t rsa -b 4096 -C "terraform" -f "$HOME/.ssh/terraform"

```



## Deploy a cluster



In order to deploy a cluster you would need to execute the following

in the `terraform` folder:



```

# prepare terraform plan, using a separate file with the credentials

$ terraform plan -var-file=.secret.aws.tfvars -out my.plan

# carry out the plan

$ terraform apply "my.plan"

```



The process should take 3-4 minutes and print out in the end something

like this:



```

aws_instance.fdb[0]: Creation complete after 3m32s (ID: i-0b90a62a90636c1ad)



Apply complete! Resources: 12 added, 0 changed, 0 destroyed.



Outputs:



fdb_address = [

    fdb-01.amazonaws.com,

    fdb-02.amazonaws.com,

    fdb-03.amazonaws.com

]

tester_address = [

    tester-01.amazonaws.com

]

```



Note that the machine names would be different each time (and much

longer). This is just a sample output.



Congratulations, you now have a FoundationDB cluster running in

AWS. You can test it by connecting to the test machine with your

terraform key:



```

$ ssh -i ~/.ssh/terraform [email protected]

```



On your first connection, the ssh might ask you about accepting the

new fingerprint. This happens because we have a brand new server

running. Just type in 'yes'.



Once connected to the test machine, you could verify that the client

tools are installed and the cluster is responding:



```

$ fdbcli

Using cluster file `/etc/foundationdb/fdb.cluster'.



The database is available.



Welcome to the fdbcli. For help, type `help'.

fdb> status details



Using cluster file `/etc/foundationdb/fdb.cluster'.



Configuration:

  Redundancy mode        - double

  Storage engine         - ssd-2

  Coordinators           - 3

  

...

```



Congratulations, FDB cluster is up and running!



# Destroy the cluster



Keeping AWS instances running costs money. So generally it is advised

to destroy all the resources after the experiment.



Terraform makes it easy:



```

$ terraform destroy -var-file=.secret.aws.tfvars



....



Plan: 0 to add, 0 to change, 12 to destroy.



Do you really want to destroy?

  Terraform will destroy all your managed infrastructure, as shown above.

  There is no undo. Only 'yes' will be accepted to confirm.



  Enter a value: yes

  

....



Destroy complete! Resources: 12 destroyed.



```



# Modify the cluster



You can tune the cluster configuration by editing `variables.tf` file

to your liking. Ideally, you would do that before creating a new

cluster.



The most important options there are:



```

variable "aws_fdb_size" {

  default = "t2.medium"

  description = "machine type to run FoundationDB servers"



}

# using only 1 machine will conflict with the default cluster config

# 'configure new memory double'

variable "aws_fdb_count" {

  default = 3

  description = "how many machines do we want in our cluster. Minimum 2"

}



variable "aws_tester_size" {

  default = "m4.xlarge"

  description = "instance type for launching tester machines"

}

```



# Plans



I plan to improve this repository a bit later by:



1. Introducing a load tester tool (pre-installed to the tester image)

   with common benchmarks.

2. Adding a few scripts to visualize the results.

3. Adding more Terraform configurations tuned for better performance.



# Contact



If you have any questions, please don't hesitate to get in touch by

sending an email to rinat at abdullin.com.