https://github.com/yandex-cloud/yc-github-loadtesting-ci

Github Actions for Yandex Cloud Load Testing
https://github.com/yandex-cloud/yc-github-loadtesting-ci
ci github-actions load-testing loadtesting yandex-cloud
Last synced: 8 months ago
JSON representation
Github Actions for Yandex Cloud Load Testing
Host: GitHub
URL: https://github.com/yandex-cloud/yc-github-loadtesting-ci
Owner: yandex-cloud
Created: 2024-06-27T11:15:49.000Z (over 1 year ago)
Default Branch: main
Last Pushed: 2024-09-26T16:32:08.000Z (about 1 year ago)
Last Synced: 2025-01-09T20:46:41.188Z (9 months ago)
Topics: ci, github-actions, load-testing, loadtesting, yandex-cloud
Language: Shell
Homepage: https://yandex.cloud/en/docs/load-testing/
Size: 65.4 KB
Stars: 1
Watchers: 8
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
Awesome Lists containing this project

README

          # GitHub Actions for Yandex Cloud Load Testing service

[USAGE EXAMPLE](.github/workflows/example.yml)

[USAGE EXAMPLE WORKFLOW](https://github.com/yandex-cloud/yc-github-loadtesting-ci/actions/workflows/example.yml)

**NOTE**: All actions require `auth-key-json` to authorize requests to Yandex Cloud services. Please check this documentation page for instructions how to create one.

## Short reference

* **test-suite** - execute multiple load tests sequentially

* **agents-create** - create and start load testing agents using Yandex Cloud Compute VMs as a hosting (Compute is billed separately)

* **agents-delete** - delete load testing agents

* **test-single-run** - execute a single load test

* **test-single-check** - check results of a single load test

## Action: `test-suite`

```yaml

uses: yandex-cloud/yc-github-loadtesting-ci/test-suite@main

```

Execute and assess a bunch of tests in a single action step.

Each test is specified by a separate directory containing:

- `test-config.yaml` test configuration file

- (optional) `meta.json` json file specifying test properties

- (optional) additional test data files which will be transferred to the agent via `data-bucket`

A list of directories containing `test-config.yaml` test configuration files should be provided via `test-directories` input parameter.

**NOTE**: Why `data-bucket` is needed? Test data files often contain sensitive information such as access tokens and customer data. Using `data-bucket` as a proxy storage guarantees that this sensitive information never gets stored elsewhere.

### Inputs

|Input|Description|Default|Required|

|-----|-----------|-------|:------:|

|`folder-id`|ID of a folder in Yandex Cloud.|n/a|yes|

|`auth-key-json-base64`|
BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json.
The action will perform all operations on behalf of a service account for which this authorized key was generated.
Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.|n/a|no|

|`auth-key-json`|Use 'auth-key-json-base64'.
An authorized key string in JSON format.
The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).|``|no|

|`action-log-level`|Action log level.|`NOTICE`|no|

|`data-bucket`|Bucket used as a proxy storage to send arbitrary test data needed for test execution to agent.
WARNING:
CI service account (authorized via auth-key-json(-base64)) must be able to upload data to this bucket
WARNING2:
agent service account must be able to download data from this bucket.|``|no|

|`artifacts-bucket`|Bucket to store artifacts generated by agent during test execution.
WARNING:
agent service account must be able to upload data to this bucket.|``|no|

|`test-directories`|A whitespace-separated list of directory paths containing test configuration files.|n/a|yes|

|`agent-filter`|A filter expression to select agents to execute tests.
Example:
  - 'name contains github' - agents with 'github' substring in name.
  - 'labels.workflow=$WORKFLOW_ID' - agents with label 'workflow' equals to $WORKFLOW_ID|`labels.workflow=${{ github.run_id }}`|no|

|`add-description`|Specified string will be added to description of created tests.|`Run from GitHub Actions`|no|

|`add-labels`|Specified labels will be added to label set of created tests. Format: 'key1=value1,key2=value2'.|`ci=github`|no|

### Outputs

|Output|Description|

|------|-----------|

|`test-ids`|IDs of performed tests.|

|`test-infos-file`|File containing a JSON array of objects with information about performed tests.|

|`test-infos`|JSON array of objects with information about performed tests.|

|`execution-report-file`|Path to generated .md execution report file.|

|`checks-report-file`|Path to generated .md checks report file.|

|`artifacts-dir`|Action artifacts directory. If needed, save it using actions/upload_artifacts.|

### Required roles

- `loadtesting.loadTester` - to create and run the test

- `storage.uploader` - to upload test data to Object Storage (required, if `data-bucket` is specified)

### Example

```yaml

- uses: actions/checkout@v4

- uses: yandex-cloud/yc-github-loadtesting-ci/test-suite@main

  with:

    folder-id: ${{ vars.YC_FOLDER_ID }}

    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

    test-directories: |-

      _impl/testdata/http-const-rps

      _impl/testdata/https-const-rps

```

## Action: `agents-create`

```yaml

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main

```

Create agents alongside with Cloud Compute VMs to host them.

  

This action is usually used used in pair with 'agents-delete' action.

**NOTE**: For security reasons you should create new service account with 'loadtesting.generatorClient' role to assign to loadtesting agent (see `service-account-id` input below) 

### Inputs

|Input|Description|Default|Required|

|-----|-----------|-------|:------:|

|`folder-id`|ID of a folder in Yandex Cloud.|n/a|yes|

|`auth-key-json-base64`|
BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json.
The action will perform all operations on behalf of a service account for which this authorized key was generated.
Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.|n/a|no|

|`auth-key-json`|Use 'auth-key-json-base64'.
An authorized key string in JSON format.
The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).|``|no|

|`action-log-level`|Action log level.|`NOTICE`|no|

|`count`|Number of agents to be created.|`1`|no|

|`service-account-id`|ID of a service account to create agent VM with.
The service account must have 'loadtesting.generatorClient' role.|``|no|

|`vm-zone`|Compute zone to create agent VM in.|`ru-central1-d`|no|

|`name-prefix`|If count=0, a name of created agent.
if count>0, a name prefix of created agents.|`onetime-ci-agent`|no|

|`description`|Description of created agents.|`Create from Github Actions`|no|

|`labels`|Labels of created agents. Format: 'key1=value1,key2=value2'.|`workflow=${{ github.run_id }}`|no|

|`cli-args`|Additional command line arguments to be passed to yc loadtesting agent create
(see yc loadtesting agent create -h for more details).
Override rules:
* '--zone ARG' overrides 'vm-zone' input parameter
* '--service-account-id ARG' overrides 'service-account-id' input parameter
* '--network-interface' overrides default network interface settings (which is - 1-to-1 NAT (dynamic public IP) in automatically chosen subnet)
* '--name ARG' overrides 'name-prefix' input parameter
* '--description ARG' overrides 'description' input parameter
* '--labels' overrides 'labels' input parameter|``|no|

|`timeout`|Time to wait for agents to become READY_FOR_TEST.
Usually, reaching this timeout means either missing permissions or invalid agent network settings.|`600`|no|

### Outputs

|Output|Description|

|------|-----------|

|`agent-ids`|IDs of created agents.|

|`artifacts-dir`|Action artifacts directory. If needed, save it using actions/upload_artifacts.|

### Required roles

- `loadtesting.loadTester` - to create an agent

- `compute.editor` - to create and start agent VM

- `iam.serviceAccounts.user` - to assign service account to agent VM

- `vpc.user` - to configure network interface on agent VM

- `vpc.publicAdmin` - to configure public IP on agent VM (the default way, may be omitted if you know what you are doing)

### Examples

**Simple:**

```yaml

# - 2 CPU, 2 GB RAM

# - one-to-one NAT in an automatically selected subnet

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main

with:

  folder-id: ${{ vars.YC_FOLDER_ID }}

  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

  service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}

  labels: "workflow=${{ github.run_id }}"

  # count: 1

  # vm-zone: ru-central1-a

```

Custom CPU and RAM settings:

```yaml

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main

with:

  folder-id: ${{ vars.YC_FOLDER_ID }}

  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

  service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}

  labels: "workflow=${{ github.run_id }}"

  cli-args: |-

    --cores 2

    --memory 2G

  # count: 1

  # vm-zone: ru-central1-a

```

Custom network settings:

This version is essentially identical to `yc loadtesting agent create ${cli-args}`.

```yaml

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main

with:

  folder-id: ${{ vars.YC_FOLDER_ID }}

  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

  cli-args: |-

    --service-account-id "${{ vars.YC_LOADTESTING_SA_ID }}"

    --labels "workflow=${{ github.run_id }}"

    --cores 2

    --memory 2G

    --zone 'ru-central1-a'

    --network-settings "subnet-name=default-a,security-group-ids=${{ vars.YC_LOADTESTING_AGENT_SECURITY_GROUP_ID }}"

  # count: 1

```

## Action: `agents-delete`

```yaml

uses: yandex-cloud/yc-github-loadtesting-ci/agents-delete@main

```

Delete agents (and the VMs).

This action is usually used used in pair with 'agents-create' action.

### Inputs

|Input|Description|Default|Required|

|-----|-----------|-------|:------:|

|`folder-id`|ID of a folder in Yandex Cloud.|n/a|yes|

|`auth-key-json-base64`|
BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json.
The action will perform all operations on behalf of a service account for which this authorized key was generated.
Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.|n/a|no|

|`auth-key-json`|Use 'auth-key-json-base64'.
An authorized key string in JSON format.
The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).|``|no|

|`action-log-level`|Action log level.|`NOTICE`|no|

|`agent-ids`|A string containing whitespace-separated list of agents to be deleted.
Normally, should be used with conjunction with agents-create.outputs.agent-ids.|n/a|yes|

### Outputs

|Output|Description|

|------|-----------|

|`artifacts-dir`|Action artifacts directory. If needed, save it using actions/upload_artifacts.|

### Required roles

- `loadtesting.loadTester` - to delete an agent

- `compute.editor` - to stop and delete agent VM

### Example

This action is usually used in pair with `agents-create` action:

```yaml

loadtesting:

  name: loadtesting job

  runs-on: ubuntu-latest

  steps:

    - id: create-agents

      uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main

      with:

        folder-id: ${{ vars.YC_FOLDER_ID }}

        auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

        service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}

    #

    # here we run some tests on created agents

    # - id: run-test

    #   ...

    #

    - id: delete-agents

      if: always() # make sure delete is called even if previous steps fail

      uses: yandex-cloud/yc-github-loadtesting-ci/agents-delete@main

      with:

        folder-id: ${{ vars.YC_FOLDER_ID }}

        auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

        # pass created agent ids to make sure they are deleted

        agent-ids: ${{ steps.agent-create.outputs.agent-ids }}

```

## Action: `test-single-run`

```yaml

uses: yandex-cloud/yc-github-loadtesting-ci/test-single-run@main

```

Execute a single test.

Each test should be specified by a separate directory containing:

- `test-config.yaml` test configuration file

- (optional) `meta.json` json file specifying test properties

- (optional) additional test data files which will be transferred to the agent via `data-bucket` 

A directory containing `test-config.yaml` test configuration file should be provided via `test-directory` input parameter.

**NOTE**: Why `data-bucket` is needed? Test data files often contain sensitive information such as access tokens and customer data. Using `data-bucket` as a proxy storage guarantees that this sensitive information never gets stored elsewhere.

### Inputs

|Input|Description|Default|Required|

|-----|-----------|-------|:------:|

|`folder-id`|ID of a folder in Yandex Cloud.|n/a|yes|

|`auth-key-json-base64`|
BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json.
The action will perform all operations on behalf of a service account for which this authorized key was generated.
Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.|n/a|no|

|`auth-key-json`|Use 'auth-key-json-base64'.
An authorized key string in JSON format.
The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).|``|no|

|`action-log-level`|Action log level.|`NOTICE`|no|

|`data-bucket`|Bucket used as a proxy storage to send arbitrary test data needed for test execution to agent.
WARNING:
CI service account (authorized via auth-key-json(-base64)) must be able to upload data to this bucket
WARNING2:
agent service account must be able to download data from this bucket.|``|no|

|`artifacts-bucket`|Bucket to store artifacts generated by agent during test execution.
WARNING:
agent service account must be able to upload data to this bucket.|``|no|

|`test-directory`|A path to a directory containing test configuration files.|n/a|yes|

|`agent-filter`|A filter expression to select agents to execute tests.
Example:
  - 'name contains github' - agents with 'github' substring in name.
  - 'labels.workflow=$WORKFLOW_ID' - agents with label 'workflow' equals to $WORKFLOW_ID|`labels.workflow=${{ github.run_id }}`|no|

|`add-description`|Specified string will be added to description of created tests.|`Run from GitHub Actions`|no|

|`add-labels`|Specified labels will be added to label set of created tests. Format: 'key1=value1,key2=value2'.|`ci=github`|no|

### Outputs

|Output|Description|

|------|-----------|

|`test-id`|ID of performed test.|

|`test-info-file`|File containing a JSON object with information about performed test.|

|`test-info`|JSON object with information about performed test.|

|`report-file`|Path to generated .md report file.|

|`artifacts-dir`|Action artifacts directory. If needed, save it using actions/upload_artifacts.|

### Required roles

- `loadtesting.loadTester` - to create and run the test

- `storage.uploader` - to upload test data to Object Storage (required, if `data-bucket` is specified)

### Example

```yaml

- uses: actions/checkout@v4

- uses: yandex-cloud/yc-github-loadtesting-ci/test-run-single@main

  with:

    folder-id: ${{ vars.YC_FOLDER_ID }}

    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

    test-directory: _impl/testdata/https-const-rps

```

## Action: `test-single-check`

```yaml

uses: yandex-cloud/yc-github-loadtesting-ci/test-single-check@main

```

Assess results measured in a single test run.

The assessment is performed via custom scripts (`check_summary.sh` and `check_report.sh`) which can be added to test directory:

```sh

TEST_ID="bb4asdfjljbhh" # some test id

TEST_DIR="_impl/testdata/http-const-rps" # test-directory

yc --format json loadtesting test get $TEST_ID > summary.json

yc --format json loadtesting test get-report-tables $TEST_ID > report.json

set -e

$DIR/check_summary.sh summary.json

$DIR/check_report.sh report.json

```

If `check_report.sh` or `check_summary.sh` are not found in test directory, some basic checks will be performed instead.

### Inputs

|Input|Description|Default|Required|

|-----|-----------|-------|:------:|

|`folder-id`|ID of a folder in Yandex Cloud.|n/a|yes|

|`auth-key-json-base64`|
BASE64 encoded authorized key string in JSON format. This setting is preferred over auth-key-json.
The action will perform all operations on behalf of a service account for which this authorized key was generated.
Given a json file, encode it via command line base64 <authorized_key.json >authorized_key.pem and add the content of result file to GitHub secrets.|n/a|no|

|`auth-key-json`|Use 'auth-key-json-base64'.
An authorized key string in JSON format.
The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).|``|no|

|`action-log-level`|Action log level.|`NOTICE`|no|

|`test-id`|ID of a test to be checked.|n/a|yes|

|`test-directory`|Directory with test configuration files.|n/a|yes|

### Outputs

|Output|Description|

|------|-----------|

|`raw-output`|Raw stdout of a check script.|

|`report-file`|Path to generated .md report file.|

|`artifacts-dir`|Action artifacts directory. If needed, save it using actions/upload_artifacts.|

### Required roles

- `loadtesting.loadTester` - to retrieve information about the test

### Example

```yaml

- uses: actions/checkout@v4

- uses: yandex-cloud/yc-github-loadtesting-ci/test-run-single@main

  id: run-test

  with:

    folder-id: ${{ vars.YC_FOLDER_ID }}

    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

    test-directory: _impl/testdata/https-const-rps

- uses: yandex-cloud/yc-github-loadtesting-ci/test-check-single@main

  with:

    folder-id: ${{ vars.YC_FOLDER_ID }}

    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

    test-id: ${{ steps.run-test.outputs.test-id }}

    test-directory: _impl/testdata/https-const-rps

```
ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/yandex-cloud/yc-github-loadtesting-ci

Awesome Lists containing this project

README
