Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/tomoyane/http-continuous-benchmarking

HTTP(S) Continuous Benchmarking tool for RESTful API. Useful for lightweight, continuous and simple measurement
https://github.com/tomoyane/http-continuous-benchmarking

benchmark benchmarking cicd continuous-benchmarking github-actions go monitoring oss performance-monitoring

Last synced: 16 days ago
JSON representation

HTTP(S) Continuous Benchmarking tool for RESTful API. Useful for lightweight, continuous and simple measurement

Host: GitHub
URL: https://github.com/tomoyane/http-continuous-benchmarking
Owner: tomoyane
License: apache-2.0
Created: 2021-02-07T14:03:48.000Z (almost 4 years ago)
Default Branch: main
Last Pushed: 2023-04-20T12:21:17.000Z (over 1 year ago)
Last Synced: 2024-10-19T03:24:15.944Z (2 months ago)
Topics: benchmark, benchmarking, cicd, continuous-benchmarking, github-actions, go, monitoring, oss, performance-monitoring
Language: Go
Homepage:
Size: 66.4 KB
Stars: 3
Watchers: 1
Forks: 1
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

        ## HTTP Continuous benchmarking

This repository provides a continuous benchmarking.  

If your project needs benchmark monitoring, it collects performance data by this repository.

|Type|Badge/URL|

|---|---|

|CI|[![ci](https://github.com/tomoyane/http-continuous-benchmarking/actions/workflows/ci.yml/badge.svg)](https://github.com/tomoyane/http-continuous-benchmarking/actions/workflows/ci.yml)|

|Release|[![release](https://github.com/tomoyane/http-continuous-benchmarking/actions/workflows/release.yml/badge.svg)](https://github.com/tomoyane/http-continuous-benchmarking/actions/workflows/release.yml)|

|Go Report Card|[![Go Report Card](https://goreportcard.com/badge/github.com/tomoyane/http-continuous-benchmarking)](https://goreportcard.com/report/github.com/tomoyane/http-continuous-benchmarking)|

|Coveralls|[![Coverage Status](https://coveralls.io/repos/github/tomoyane/http-continuous-benchmarking/badge.svg?branch=main)](https://coveralls.io/github/tomoyane/http-continuous-benchmarking?branch=main)|

|Coverage report for GitHub Pages|[Coverage report GitHub Pages](https://tomoyane.github.io/http-continuous-benchmarking/#file0)

|Docker Registry|[http-continuous-benchmarking](https://hub.docker.com/r/tomohito/http-continuous-benckmarking)|

### Concept

* Simple and lightweight benchmark tool

  * This tool is not recommended for if large-scale performance measurements

* Continuous benchmarking tool

  * Always monitor performance impacts for source code changes

* Lightweight to make various requests

  * The user does not do complicated things

* Create measured reports and notify warnings

  * Create a continuous report and associate it with a commit hash

  * Warn and notice when the threshold is reached

### How to use GitHub Actions

With input variable.  

**(※)** is required.

|With input name|Description|Example|

|---|---|---|

|target_url **(※)**|Request destination URL.|http(s)://xxxxxxx.com/api/v1/users|

|http_headers **(※)**|Request HTTP Headers. Json `{}` format.|{"Authorization": "Bearer xxx", "Content-Type": "application/json"}|

|thread_num **(※)**|Client thread num.|5|

|trial_num **(※)**|Benchmark trial number while 5seconds. If its 5times, the benchmark try 5times * 5seconds.|5 
(Ex: Case of API 100rps, 100(rps) * 5(seconds) * 5(times))|

|req_http_method_ratio **(※)**|HTTP method percentage of request. Json `{}` format.|{"POST": 4, "GET": 6}|

|req_body|HTTP Request Body. If you use PUT or PATCH or POST, its required. Json `{}` format.|{"email": "[email protected]"}|

|enable_alert|Alert. You want to catch alert when over threshold.|true or false|

|slack_web_hook_url|Slack web hook url. You can get [here](https://moff-bear.slack.com/apps/new/A0F7XDUAZ--incoming-webhook-)|https://hooks.slack.com/services/XXXXXXX/XXXXXXX/XXXXXXXXXXXXXX|

|slack_channel|Slack channel.|test-01|

|slack_notify_threshold_latency_millis|Threshold latency milliseconds for notification. This is to check latency average.|100|

|slack_notify_threshold_rps|Threshold request per seconds for notification. |20|

Sample GitHub actions workflow yaml.

```yaml

on: [push]

jobs:

  benchmarking:

    runs-on: ubuntu-latest

    name: Attack

    steps:

      - name: Benchmarking

        id: benchmarking

        uses: tomoyane/[email protected]

        with:

          target_url: 'https://example.com'

          http_headers: '{"Content-Type":"application/json"}'

          thread_num: '1'

          trial_num: '1'

          req_http_method_ratio: '{"GET": 10}'

      - name: Completed

        run: echo "Completed benchmarking"

```

With Slack notification workflow yaml.

```yaml

on: [push]

jobs:

  benchmarking:

    runs-on: ubuntu-latest

    name: Attack

    steps:

      - name: Benchmarking

        id: benchmarking

        uses: tomoyane/[email protected]

        with:

          target_url: 'https://example.com'

          http_headers: '{"Content-Type":"application/json"}'

          thread_num: '1'

          trial_num: '1'

          req_http_method_ratio: '{"GET": 10}'

          enable_alert: 'true'

          slack_web_hook_url: 'https://hooks.slack.com/services/XXXXXXX/XXXXXXX/XXXXXXXXXXXXXX'

          slack_channel: 'test-01'

          slack_notify_threshold_latency_millis: 20

      - name: Completed

        run: echo "Completed benchmarking"

```

If result is over threshold, you will catch bellow message.

![slack_msg_sample](https://user-images.githubusercontent.com/9509132/147807558-91190c09-a2d8-45cc-b63a-66b0dbd7b131.png)

### How to use simple application

[This repository application usage](https://github.com/tomoyane/http-continuous-benchmarking)

Application basic usage.

```bash

# Set required environment variable before execution

$ git clone https://github.com/tomoyane/http-continuous-benchmarking.git

$ cd http-continuous-benchmarking; go build

$ export INPUT_TARGET_URL='https://example.com' \

         INPUT_REQ_HTTP_METHOD_RATIO='{"GET":10}' \

         INPUT_HTTP_HEADERS='{"Content-Type":"application/json"}' \

         INPUT_THREAD_NUM=2 \

         INPUT_TRIAL_NUM=2

$ ./http-continuous-benchmarking

## With Alert to Slack

$ export INPUT_TARGET_URL='https://example.com' \

         INPUT_REQ_HTTP_METHOD_RATIO='{"GET":10}' \

         INPUT_HTTP_HEADERS='{"Content-Type":"application/json"}' \

         INPUT_THREAD_NUM=2 \

         INPUT_TRIAL_NUM=2 \

         INPUT_ENABLE_ALERT=true \

         INPUT_SLACK_WEB_HOOK_URL="https://hooks.slack.com/services/XXXXXXX/XXXXXXX/XXXXXXXXXXXXXX" \

         INPUT_SLACK_CHANNEL="test-01" \

         INPUT_SLACK_NOTIFY_THRESHOLD_LATENCY_MILLIS=100 # Case of latency

$ ./http-continuous-benchmarking

```

Docker image usage.

```bash

$ docker pull tomohito/http-continuous-benckmarking

$ docker run \

  --env INPUT_TARGET_URL='https://example.com' \

  --env INPUT_REQ_HTTP_METHOD_RATIO='{"GET":10}' \

  --env INPUT_HTTP_HEADERS='{"Content-Type":"application/json"}' \

  --env INPUT_THREAD_NUM=2 \

  --env INPUT_TRIAL_NUM=2 \

  -i tomohito/http-continuous-benckmarking

```

### Example benchmark result

#### Metrics report HTML

![Screen Shot 2021-02-28 at 11 16 19](https://user-images.githubusercontent.com/9509132/109417530-656e0180-7a07-11eb-922a-e6915d194eb8.png)

#### Log

```bash

$ ./http-continuous-benchmarking

HTTP request pattern according to the ratio = GET GET GET GET GET GET GET GET GET GET

Start warnmup for duration 5 seconds

End warmup

Start time = 1614653873

(Thread-3): Start attack for duration 5 seconds

(Thread-1): Start attack for duration 5 seconds

(Thread-2): Start attack for duration 5 seconds

(Thread-1): End attack

(Thread-2): End attack

(Thread-3): End attack

GET request stats information

Latency 99  percentile: 190 milliseconds

Latency 95  percentile: 172 milliseconds

Latency avg percentile: 115 milliseconds

Latency max percentile: 210 milliseconds

Latency min percentile: 72 milliseconds

Request per seconds:    30

(Thread-1): Start attack for duration 5 seconds

(Thread-3): Start attack for duration 5 seconds

(Thread-2): Start attack for duration 5 seconds

(Thread-1): End attack

(Thread-2): End attack

(Thread-3): End attack

GET request stats information

Latency 99  percentile: 153 milliseconds

Latency 95  percentile: 129 milliseconds

Latency avg percentile: 94 milliseconds

Latency max percentile: 192 milliseconds

Latency min percentile: 65 milliseconds

Request per seconds:    36

End time = 1614653894

```