{"id":31831232,"url":"https://github.com/livingdocsio/performance-testing","last_synced_at":"2025-10-11T21:48:26.918Z","repository":{"id":68391061,"uuid":"136044481","full_name":"livingdocsIO/performance-testing","owner":"livingdocsIO","description":null,"archived":false,"fork":false,"pushed_at":"2023-12-15T08:33:48.000Z","size":36,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-04-14T11:56:02.853Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/livingdocsIO.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-06-04T15:19:36.000Z","updated_at":"2024-04-14T11:56:02.854Z","dependencies_parsed_at":"2023-09-13T04:35:11.352Z","dependency_job_id":null,"html_url":"https://github.com/livingdocsIO/performance-testing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/livingdocsIO/performance-testing","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livingdocsIO%2Fperformance-testing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livingdocsIO%2Fperformance-testing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livingdocsIO%2Fperformance-testing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livingdocsIO%2Fperformance-testing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/livingdocsIO","download_url":"https://codeload.github.com/livingdocsIO/performance-testing/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livingdocsIO%2Fperformance-testing/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279008824,"owners_count":26084518,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-11T21:48:24.961Z","updated_at":"2025-10-11T21:48:26.910Z","avatar_url":"https://github.com/livingdocsIO.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# What is this project about\n\nWhen we do performance or load testing, we usually go through the following steps:\n  1. prepare urls following the process outlined [here](https://github.com/livingdocsIO/team/blob/master/Coding/Performance/prepare-urls.md).\n  2. provision aws machines with node and artillery.\n  3. distribute the artillery configuration to the aws machines.\n  4. execute the artillery script against the aws machines which in turn make requests against the target server.\n\nThis project takes care of the tedious task to provision the aws machines, distribute the artillery configuration to the aws machines and executes the artillery scripts on all machines via ansible playbooks.\n\n# Prerequisites\n\n## Ansible\n\n- please install it through `pip` (not through `brew`). Ansible is using `boto` (python interface for AWS), so installing ansible through `pip` is the working option. Something like the following should work (your mileage may vary):\n\n  ```\n  $ pip install --upgrade pip\n  $ pip install boto\n  $ pip install ansible\n  $ ansible --version\n  ansible 2.5.4\n    config file = /Users/georgi/Projects/livingdocs/bluewin/performance-testing/ansible.cfg\n    configured module search path = ['/Users/georgi/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']\n    ansible python module location = /Users/georgi/.pyenv/versions/3.5.0/lib/python3.5/site-packages/ansible\n    executable location = /Users/georgi/.pyenv/versions/3.5.0/bin/ansible\n    python version = 3.5.0 (default, Mar 17 2017, 17:54:06) [GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)]\n  ```\n\n## Terraform\n\n- installation guide if you need one: https://www.terraform.io/downloads.html.\n- example of terraform running successfully:\n\n  ```\n  $ terraform --version\n  Terraform v0.11.7\n  ```\n\n## AWS cli\n\n- you need to configure the AWS cli. This might be already configured for most of you, but make sure you use the right credentials (bluewin, livingdocs) as this setup would likely incur some charges on your bill.\n- the project needs aws access key and aws secret access key.\n- here is a good [aws + terraform guide](https://hackernoon.com/introduction-to-aws-with-terraform-7a8daf261dc0). Use the part on how to create on aws account and configure the cli.\n- example of a successful configuration:\n\n  ```\n  $ aws --version\n  aws-cli/1.15.31 Python/3.5.0 Darwin/17.5.0 botocore/1.10.31\n  $ aws configure list\n        Name                    Value             Type    Location\n        ----                    -----             ----    --------\n     profile                \u003cnot set\u003e             None    None\n  access_key     ****************57VQ shared-credentials-file\n  secret_key     ****************ANQ1 shared-credentials-file\n      region             eu-central-1      config-file    ~/.aws/config\n  ```\n\n\n## Launched instances\n\n- the terraform setup prepares machines to make requests with `artillery` against the target servers. You can learn more about artillery here: https://artillery.io/.\n- all provisioned machines are of the same ec2 instance type. However one of them is tagged with `mobile` and the others are tagged with `web`. Later we would use this notation to deliver the right artillery configuration. \n- We employ different artillery configuration to make a regular `GET` request or a mobile api request with custom headers (this is the bluewin specific part).\n- by default we launch 2 instances (one `web` and one `mobile`). The `web` instance would make a regular `GET` request. The `mobile` instance would make a request with custom header which would deliver json.\n- you can launch any number of instances (for example `10`) using the following command:\n\n```\nterraform apply -auto-approve -var count=10\n```\n\n- the above command will launch 10 instances: 9 `web` instances and 1 `mobile` instance.\n- have a look at [terraform.tfvars](./terraform.tfvars) for a list of variables you can override during terraform setup.\n\n# Setup\n\n## Terraform\n\n1. You need to setup terraform and install the needed packages:\n\n```\n$ terraform init\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eFull output of a successful run\u003c/summary\u003e\n\n```bash\nInitializing provider plugins...\n- Checking for available provider plugins on https://releases.hashicorp.com...\n- Downloading plugin for provider \"aws\" (1.22.0)...\n\nThe following providers do not have any version constraints in configuration,\nso the latest version was installed.\n\nTo prevent automatic upgrades to new major versions that may contain breaking\nchanges, it is recommended to add version = \"...\" constraints to the\ncorresponding provider blocks in configuration, with the constraint strings\nsuggested below.\n\n* provider.aws: version = \"~\u003e 1.22\"\n\nTerraform has been successfully initialized!\n\nYou may now begin working with Terraform. Try running \"terraform plan\" to see\nany changes that are required for your infrastructure. All Terraform commands\nshould now work.\n\nIf you ever set or change modules or backend configuration for Terraform,\nrerun this command to reinitialize your working directory. If you forget, other\ncommands will detect it and remind you to do so if necessary\n```\n\u003c/details\u003e\n\n\u003cbr/\u003e\n\n2. Ensure you have proper aws configuration, connection and terraform setup by checking the state:\n\n```\nterraform plan\n```\n\nIf aws and terraform setup is correct and working, you would most likely see an output which at the end says:\n\n```\nPlan: 4 to add, 0 to change, 0 to destroy.\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eFull output of a successful run\u003c/summary\u003e\n\n```bash\n$ terraform plan\nRefreshing Terraform state in-memory prior to plan...\nThe refreshed state will be used to calculate this plan, but will not be\npersisted to local or remote state storage.\n\n\n------------------------------------------------------------------------\n\nAn execution plan has been generated and is shown below.\nResource actions are indicated with the following symbols:\n  + create\n\nTerraform will perform the following actions:\n\n  + aws_default_vpc.default\n      id:                                    \u003ccomputed\u003e\n      assign_generated_ipv6_cidr_block:      \u003ccomputed\u003e\n      cidr_block:                            \u003ccomputed\u003e\n      default_network_acl_id:                \u003ccomputed\u003e\n      default_route_table_id:                \u003ccomputed\u003e\n      default_security_group_id:             \u003ccomputed\u003e\n      dhcp_options_id:                       \u003ccomputed\u003e\n      enable_classiclink:                    \u003ccomputed\u003e\n      enable_classiclink_dns_support:        \u003ccomputed\u003e\n      enable_dns_hostnames:                  \u003ccomputed\u003e\n      enable_dns_support:                    \"true\"\n      instance_tenancy:                      \u003ccomputed\u003e\n      ipv6_association_id:                   \u003ccomputed\u003e\n      ipv6_cidr_block:                       \u003ccomputed\u003e\n      main_route_table_id:                   \u003ccomputed\u003e\n\n  + aws_instance.web-performance-test[0]\n      id:                                    \u003ccomputed\u003e\n      ami:                                   \"ami-c7e0c82c\"\n      associate_public_ip_address:           \u003ccomputed\u003e\n      availability_zone:                     \u003ccomputed\u003e\n      ebs_block_device.#:                    \u003ccomputed\u003e\n      ephemeral_block_device.#:              \u003ccomputed\u003e\n      get_password_data:                     \"false\"\n      instance_state:                        \u003ccomputed\u003e\n      instance_type:                         \"t2.micro\"\n      ipv6_address_count:                    \u003ccomputed\u003e\n      ipv6_addresses.#:                      \u003ccomputed\u003e\n      key_name:                              \"terraform\"\n      network_interface.#:                   \u003ccomputed\u003e\n      network_interface_id:                  \u003ccomputed\u003e\n      password_data:                         \u003ccomputed\u003e\n      placement_group:                       \u003ccomputed\u003e\n      primary_network_interface_id:          \u003ccomputed\u003e\n      private_dns:                           \u003ccomputed\u003e\n      private_ip:                            \u003ccomputed\u003e\n      public_dns:                            \u003ccomputed\u003e\n      public_ip:                             \u003ccomputed\u003e\n      root_block_device.#:                   \u003ccomputed\u003e\n      security_groups.#:                     \"1\"\n      security_groups.1271829205:            \"with-ssh-security-group\"\n      source_dest_check:                     \"true\"\n      subnet_id:                             \u003ccomputed\u003e\n      tags.%:                                \"1\"\n      tags.Name:                             \"web-performance-test\"\n      tenancy:                               \u003ccomputed\u003e\n      user_data:                             \"8ad701e23aad06df493cc1291465222a8155b4c5\"\n      volume_tags.%:                         \u003ccomputed\u003e\n      vpc_security_group_ids.#:              \u003ccomputed\u003e\n\n  + aws_instance.web-performance-test[1]\n      id:                                    \u003ccomputed\u003e\n      ami:                                   \"ami-c7e0c82c\"\n      associate_public_ip_address:           \u003ccomputed\u003e\n      availability_zone:                     \u003ccomputed\u003e\n      ebs_block_device.#:                    \u003ccomputed\u003e\n      ephemeral_block_device.#:              \u003ccomputed\u003e\n      get_password_data:                     \"false\"\n      instance_state:                        \u003ccomputed\u003e\n      instance_type:                         \"t2.micro\"\n      ipv6_address_count:                    \u003ccomputed\u003e\n      ipv6_addresses.#:                      \u003ccomputed\u003e\n      key_name:                              \"terraform\"\n      network_interface.#:                   \u003ccomputed\u003e\n      network_interface_id:                  \u003ccomputed\u003e\n      password_data:                         \u003ccomputed\u003e\n      placement_group:                       \u003ccomputed\u003e\n      primary_network_interface_id:          \u003ccomputed\u003e\n      private_dns:                           \u003ccomputed\u003e\n      private_ip:                            \u003ccomputed\u003e\n      public_dns:                            \u003ccomputed\u003e\n      public_ip:                             \u003ccomputed\u003e\n      root_block_device.#:                   \u003ccomputed\u003e\n      security_groups.#:                     \"1\"\n      security_groups.1271829205:            \"with-ssh-security-group\"\n      source_dest_check:                     \"true\"\n      subnet_id:                             \u003ccomputed\u003e\n      tags.%:                                \"1\"\n      tags.Name:                             \"web-performance-test\"\n      tenancy:                               \u003ccomputed\u003e\n      user_data:                             \"8ad701e23aad06df493cc1291465222a8155b4c5\"\n      volume_tags.%:                         \u003ccomputed\u003e\n      vpc_security_group_ids.#:              \u003ccomputed\u003e\n\n  + aws_security_group.with_ssh\n      id:                                    \u003ccomputed\u003e\n      arn:                                   \u003ccomputed\u003e\n      description:                           \"Managed by Terraform\"\n      egress.#:                              \"1\"\n      egress.482069346.cidr_blocks.#:        \"1\"\n      egress.482069346.cidr_blocks.0:        \"0.0.0.0/0\"\n      egress.482069346.description:          \"\"\n      egress.482069346.from_port:            \"0\"\n      egress.482069346.ipv6_cidr_blocks.#:   \"0\"\n      egress.482069346.prefix_list_ids.#:    \"0\"\n      egress.482069346.protocol:             \"-1\"\n      egress.482069346.security_groups.#:    \"0\"\n      egress.482069346.self:                 \"false\"\n      egress.482069346.to_port:              \"0\"\n      ingress.#:                             \"1\"\n      ingress.2541437006.cidr_blocks.#:      \"1\"\n      ingress.2541437006.cidr_blocks.0:      \"0.0.0.0/0\"\n      ingress.2541437006.description:        \"\"\n      ingress.2541437006.from_port:          \"22\"\n      ingress.2541437006.ipv6_cidr_blocks.#: \"0\"\n      ingress.2541437006.protocol:           \"tcp\"\n      ingress.2541437006.security_groups.#:  \"0\"\n      ingress.2541437006.self:               \"false\"\n      ingress.2541437006.to_port:            \"22\"\n      name:                                  \"with-ssh-security-group\"\n      owner_id:                              \u003ccomputed\u003e\n      revoke_rules_on_delete:                \"false\"\n      vpc_id:                                \"${aws_default_vpc.default.id}\"\n\n\nPlan: 4 to add, 0 to change, 0 to destroy.\n\n------------------------------------------------------------------------\n\nNote: You didn't specify an \"-out\" parameter to save this plan, so Terraform\ncan't guarantee that exactly these actions will be performed if\n\"terraform apply\" is subsequently run.\n```\n\n\u003c/details\u003e\n\n## Create a new key pair\n\n- we need a key pair so we could later connect to the machines via ssh.\n- create a new key pait (take from https://docs.aws.amazon.com/cli/latest/userguide/cli-ec2-keypairs.html):\n\n  ```\n  aws ec2 create-key-pair --key-name terraform --query 'KeyMaterial' --output text \u003e terraform.pem \u0026\u0026 chmod 400 terraform.pem\n  ```\n\n- the key pair we have in the [variables](./variables.tfvars) and used throughout this README is named `terraform`. If you want to name your ssh key differently, make sure you pass the `key_name`:\n\n  ```\n  terraform apply -auto-approve -var key_name=\"MY_SSH_KEY_NAME\"\n  ```\n\n## Prepare urls list\n\n- the urls list is a simple file with only one url per line.\n- here we don't tackle the question how we obtain the [urls list](./provision/urls.txt).\n- steps how this list could be prepared for bluewin specific scenario could be found [here](https://github.com/livingdocsIO/team/blob/master/Coding/Performance/prepare-urls.md).\n\n# Usage\n\n1. Plan (get a preview of what would be the impact of the command - what would be created, deleted and/or updated):\n\n```\nterraform plan\n```\n\n2. Apply\n\n```\nterraform apply\n\nDo you want to perform these actions?\n  Terraform will perform the actions described above.\n  Only 'yes' will be accepted to approve.\n\n  Enter a value:\n```\n\nSelect `yes` when asked to enter a value.\n\nIf you know what you are doing and you want to skip the confirmation:\n\n```\nterraform apply -auto-approve\n```\n\n3. Configure artillery (uploads artillery templates and urls list to the remote hosts):\n\n```\nansible-playbook ./configure-artillery.yml\n```\n\n4. Perform a test:\n\n- with `artillery_duration` you control for how long (in seconds) to run the performance test. \n- with `artillery_arrival_rate` you control for how many requests per second a **single instance** should make. \n- for example if you want to have 100 requests per second for 1 minute and you have 10 instances, you would use:\n\n  ```\n  ansible-playbook perform-test.yml --extra-vars \"artillery_duration=60 artillery_arrival_rate=10\"\n  ```\n\n# Results\n\n- once you run the load test you can have a look at the results in the `results` folder:\n\n  ![screen shot 2018-06-07 at 15 54 39](https://user-images.githubusercontent.com/1632188/41103871-787b7802-6a6a-11e8-92d6-34395e1e80dd.png)\n\n- filename legend for `ec2-18-184-248-107.eu-central-1.compute.amazonaws.com-report-10-rps.html`:\n  - the public dns of the instance (for example: `ec2-18-184-248-107.eu-central-1.compute.amazonaws.com`)\n  - the (total) number of requests during this run of the performance testing - `10-rps`.\n  - you get the `.json` and the `.html` reports.\n  - if you open the `.html` report in a browser, it gives you a good idea how the load testing went.\n\n# Clean up\n\n\n- once you are done with the load testing and the results are collected, you should **remove all instances and the ssh key pair**:\n\n```\nterraform destroy -auto-approve \u0026\u0026 aws ec2 delete-key-pair --key-name terraform \u0026\u0026 rm -f terraform.pem\n```\n\n# Caveats\n\n- there is no live output of artillery while executing load testing the tested server. Ansible has not yet implemented it as a feature. Read [here](https://github.com/ansible/ansible/issues/3887) and [here](https://github.com/ansible/ansible/issues/4870).\n\n# Help and support\n\n## Displaying Your Key Pair\n\n```\naws ec2 describe-key-pairs --key-name terraform\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flivingdocsio%2Fperformance-testing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flivingdocsio%2Fperformance-testing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flivingdocsio%2Fperformance-testing/lists"}