https://github.com/amirhnajafiz-archive/natsbench-runbook

A runbook for better NATS benchmarking.
https://github.com/amirhnajafiz-archive/natsbench-runbook

nats nats-bench nats-cli nats-jetstream python python3

Last synced: 4 months ago
JSON representation

A runbook for better NATS benchmarking.

• Host: GitHub
• URL: https://github.com/amirhnajafiz-archive/natsbench-runbook
• Owner: amirhnajafiz
• Created: 2021-04-03T17:21:19.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2024-06-04T06:29:45.000Z (over 1 year ago)
• Last Synced: 2024-06-04T07:44:14.128Z (over 1 year ago)
• Topics: nats, nats-bench, nats-cli, nats-jetstream, python, python3
• Language: Jupyter Notebook
• Homepage:
• Size: 313 KB
• Stars: 6
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# NATS Bench Runbook

![GitHub top language](https://img.shields.io/github/languages/top/amirhnajafiz/natsbench-runbook)

![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/amirhnajafiz/natsbench-runbook)

![GitHub Tag](https://img.shields.io/github/v/tag/amirhnajafiz/natsbench-runbook)

This is a runbook for automating nats-bench commands. It uses nats-cli bench feature in order to execute bench commands.

## Config

Copy the example ```cmd.example.json``` file in **config** directory into ```cmd.json``` in root directory.After that you can change the commands in that json file. After running ```python3 main.py``` command, it will read your commands from ```cmd.json``` file and exports the results into ```tmp``` directory.

A complete section of a command is represented below. Each command has a ```name``` lable. The ```count``` label is used when you want to repeat a command multiple times. ```command``` field is where you put nats-bench attributes (please don't as ```--csv``` and ```--no-progress``` commands since the runbook sets them). The ```pre-commands``` fields is an array of commands that you want to run before bench process. If you set ```syscall``` label, you can run any shell commands instead of nats-bench commands.

```json

[

    {

        "name": "context-switch",

        "command": "nats context select default",

        "syscall": true

    },

    {

        "name": "normal-stats",

        "count": 5,

        "command": "rides --pub 5 --sub 15",

        "pre-commands": [

            "nats stream delete benchstream --force"

        ],

        "syscall": false

    }

]

```

### context config

```json

{

  "description": "",

  "url": "nats://172.21.88.124:4222",

  "socks_proxy": "",

  "token": "",

  "user": "",

  "password": "",

  "creds": "",

  "nkey": "",

  "cert": "",

  "key": "",

  "ca": "",

  "nsc": "",

  "jetstream_domain": "",

  "jetstream_api_prefix": "",

  "jetstream_event_prefix": "",

  "inbox_prefix": "",

  "user_jwt": "",

  "color_scheme": ""

}

```

## Hypothesis testing

A good feature in this runbook is hypothesis testing of a config change in nats-server. After you benchmakr the cluster, you can compare your changes by running ```python3 htest.py``` command. This script gets two input argument which are the directory names in ```tmp``` directory (referred as groups). After that, it will run a hypothesis test to compare read, write, and overall stats between those groups.

## Build and run

```sh

docker build . -f build/Dockerfile -t amirhossein21/natsbench-runbook:v0.0.4

docker run -v ./tmp:/usr/src/app/tmp -v ./cmd.json:/usr/src/app/cmd.json -v ./default.json:/root/.config/nats/context/js-p.json amirhossein21/natsbench-runbook:v0.0.4 nats context ls

```