https://github.com/livingdocsio/performance-testing
https://github.com/livingdocsio/performance-testing
Last synced: 8 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/livingdocsio/performance-testing
- Owner: livingdocsIO
- Created: 2018-06-04T15:19:36.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2023-12-15T08:33:48.000Z (over 2 years ago)
- Last Synced: 2024-04-14T11:56:02.853Z (about 2 years ago)
- Language: Python
- Size: 35.2 KB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# What is this project about
When we do performance or load testing, we usually go through the following steps:
1. prepare urls following the process outlined [here](https://github.com/livingdocsIO/team/blob/master/Coding/Performance/prepare-urls.md).
2. provision aws machines with node and artillery.
3. distribute the artillery configuration to the aws machines.
4. execute the artillery script against the aws machines which in turn make requests against the target server.
This project takes care of the tedious task to provision the aws machines, distribute the artillery configuration to the aws machines and executes the artillery scripts on all machines via ansible playbooks.
# Prerequisites
## Ansible
- please install it through `pip` (not through `brew`). Ansible is using `boto` (python interface for AWS), so installing ansible through `pip` is the working option. Something like the following should work (your mileage may vary):
```
$ pip install --upgrade pip
$ pip install boto
$ pip install ansible
$ ansible --version
ansible 2.5.4
config file = /Users/georgi/Projects/livingdocs/bluewin/performance-testing/ansible.cfg
configured module search path = ['/Users/georgi/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Users/georgi/.pyenv/versions/3.5.0/lib/python3.5/site-packages/ansible
executable location = /Users/georgi/.pyenv/versions/3.5.0/bin/ansible
python version = 3.5.0 (default, Mar 17 2017, 17:54:06) [GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)]
```
## Terraform
- installation guide if you need one: https://www.terraform.io/downloads.html.
- example of terraform running successfully:
```
$ terraform --version
Terraform v0.11.7
```
## AWS cli
- you need to configure the AWS cli. This might be already configured for most of you, but make sure you use the right credentials (bluewin, livingdocs) as this setup would likely incur some charges on your bill.
- the project needs aws access key and aws secret access key.
- here is a good [aws + terraform guide](https://hackernoon.com/introduction-to-aws-with-terraform-7a8daf261dc0). Use the part on how to create on aws account and configure the cli.
- example of a successful configuration:
```
$ aws --version
aws-cli/1.15.31 Python/3.5.0 Darwin/17.5.0 botocore/1.10.31
$ aws configure list
Name Value Type Location
---- ----- ---- --------
profile None None
access_key ****************57VQ shared-credentials-file
secret_key ****************ANQ1 shared-credentials-file
region eu-central-1 config-file ~/.aws/config
```
## Launched instances
- the terraform setup prepares machines to make requests with `artillery` against the target servers. You can learn more about artillery here: https://artillery.io/.
- all provisioned machines are of the same ec2 instance type. However one of them is tagged with `mobile` and the others are tagged with `web`. Later we would use this notation to deliver the right artillery configuration.
- We employ different artillery configuration to make a regular `GET` request or a mobile api request with custom headers (this is the bluewin specific part).
- by default we launch 2 instances (one `web` and one `mobile`). The `web` instance would make a regular `GET` request. The `mobile` instance would make a request with custom header which would deliver json.
- you can launch any number of instances (for example `10`) using the following command:
```
terraform apply -auto-approve -var count=10
```
- the above command will launch 10 instances: 9 `web` instances and 1 `mobile` instance.
- have a look at [terraform.tfvars](./terraform.tfvars) for a list of variables you can override during terraform setup.
# Setup
## Terraform
1. You need to setup terraform and install the needed packages:
```
$ terraform init
```
Full output of a successful run
```bash
Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "aws" (1.22.0)...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
* provider.aws: version = "~> 1.22"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary
```
2. Ensure you have proper aws configuration, connection and terraform setup by checking the state:
```
terraform plan
```
If aws and terraform setup is correct and working, you would most likely see an output which at the end says:
```
Plan: 4 to add, 0 to change, 0 to destroy.
```
Full output of a successful run
```bash
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ aws_default_vpc.default
id:
assign_generated_ipv6_cidr_block:
cidr_block:
default_network_acl_id:
default_route_table_id:
default_security_group_id:
dhcp_options_id:
enable_classiclink:
enable_classiclink_dns_support:
enable_dns_hostnames:
enable_dns_support: "true"
instance_tenancy:
ipv6_association_id:
ipv6_cidr_block:
main_route_table_id:
+ aws_instance.web-performance-test[0]
id:
ami: "ami-c7e0c82c"
associate_public_ip_address:
availability_zone:
ebs_block_device.#:
ephemeral_block_device.#:
get_password_data: "false"
instance_state:
instance_type: "t2.micro"
ipv6_address_count:
ipv6_addresses.#:
key_name: "terraform"
network_interface.#:
network_interface_id:
password_data:
placement_group:
primary_network_interface_id:
private_dns:
private_ip:
public_dns:
public_ip:
root_block_device.#:
security_groups.#: "1"
security_groups.1271829205: "with-ssh-security-group"
source_dest_check: "true"
subnet_id:
tags.%: "1"
tags.Name: "web-performance-test"
tenancy:
user_data: "8ad701e23aad06df493cc1291465222a8155b4c5"
volume_tags.%:
vpc_security_group_ids.#:
+ aws_instance.web-performance-test[1]
id:
ami: "ami-c7e0c82c"
associate_public_ip_address:
availability_zone:
ebs_block_device.#:
ephemeral_block_device.#:
get_password_data: "false"
instance_state:
instance_type: "t2.micro"
ipv6_address_count:
ipv6_addresses.#:
key_name: "terraform"
network_interface.#:
network_interface_id:
password_data:
placement_group:
primary_network_interface_id:
private_dns:
private_ip:
public_dns:
public_ip:
root_block_device.#:
security_groups.#: "1"
security_groups.1271829205: "with-ssh-security-group"
source_dest_check: "true"
subnet_id:
tags.%: "1"
tags.Name: "web-performance-test"
tenancy:
user_data: "8ad701e23aad06df493cc1291465222a8155b4c5"
volume_tags.%:
vpc_security_group_ids.#:
+ aws_security_group.with_ssh
id:
arn:
description: "Managed by Terraform"
egress.#: "1"
egress.482069346.cidr_blocks.#: "1"
egress.482069346.cidr_blocks.0: "0.0.0.0/0"
egress.482069346.description: ""
egress.482069346.from_port: "0"
egress.482069346.ipv6_cidr_blocks.#: "0"
egress.482069346.prefix_list_ids.#: "0"
egress.482069346.protocol: "-1"
egress.482069346.security_groups.#: "0"
egress.482069346.self: "false"
egress.482069346.to_port: "0"
ingress.#: "1"
ingress.2541437006.cidr_blocks.#: "1"
ingress.2541437006.cidr_blocks.0: "0.0.0.0/0"
ingress.2541437006.description: ""
ingress.2541437006.from_port: "22"
ingress.2541437006.ipv6_cidr_blocks.#: "0"
ingress.2541437006.protocol: "tcp"
ingress.2541437006.security_groups.#: "0"
ingress.2541437006.self: "false"
ingress.2541437006.to_port: "22"
name: "with-ssh-security-group"
owner_id:
revoke_rules_on_delete: "false"
vpc_id: "${aws_default_vpc.default.id}"
Plan: 4 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.
```
## Create a new key pair
- we need a key pair so we could later connect to the machines via ssh.
- create a new key pait (take from https://docs.aws.amazon.com/cli/latest/userguide/cli-ec2-keypairs.html):
```
aws ec2 create-key-pair --key-name terraform --query 'KeyMaterial' --output text > terraform.pem && chmod 400 terraform.pem
```
- the key pair we have in the [variables](./variables.tfvars) and used throughout this README is named `terraform`. If you want to name your ssh key differently, make sure you pass the `key_name`:
```
terraform apply -auto-approve -var key_name="MY_SSH_KEY_NAME"
```
## Prepare urls list
- the urls list is a simple file with only one url per line.
- here we don't tackle the question how we obtain the [urls list](./provision/urls.txt).
- steps how this list could be prepared for bluewin specific scenario could be found [here](https://github.com/livingdocsIO/team/blob/master/Coding/Performance/prepare-urls.md).
# Usage
1. Plan (get a preview of what would be the impact of the command - what would be created, deleted and/or updated):
```
terraform plan
```
2. Apply
```
terraform apply
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value:
```
Select `yes` when asked to enter a value.
If you know what you are doing and you want to skip the confirmation:
```
terraform apply -auto-approve
```
3. Configure artillery (uploads artillery templates and urls list to the remote hosts):
```
ansible-playbook ./configure-artillery.yml
```
4. Perform a test:
- with `artillery_duration` you control for how long (in seconds) to run the performance test.
- with `artillery_arrival_rate` you control for how many requests per second a **single instance** should make.
- for example if you want to have 100 requests per second for 1 minute and you have 10 instances, you would use:
```
ansible-playbook perform-test.yml --extra-vars "artillery_duration=60 artillery_arrival_rate=10"
```
# Results
- once you run the load test you can have a look at the results in the `results` folder:
- filename legend for `ec2-18-184-248-107.eu-central-1.compute.amazonaws.com-report-10-rps.html`:
- the public dns of the instance (for example: `ec2-18-184-248-107.eu-central-1.compute.amazonaws.com`)
- the (total) number of requests during this run of the performance testing - `10-rps`.
- you get the `.json` and the `.html` reports.
- if you open the `.html` report in a browser, it gives you a good idea how the load testing went.
# Clean up
- once you are done with the load testing and the results are collected, you should **remove all instances and the ssh key pair**:
```
terraform destroy -auto-approve && aws ec2 delete-key-pair --key-name terraform && rm -f terraform.pem
```
# Caveats
- there is no live output of artillery while executing load testing the tested server. Ansible has not yet implemented it as a feature. Read [here](https://github.com/ansible/ansible/issues/3887) and [here](https://github.com/ansible/ansible/issues/4870).
# Help and support
## Displaying Your Key Pair
```
aws ec2 describe-key-pairs --key-name terraform
```