{"id":15404698,"url":"https://github.com/k1low/runn","last_synced_at":"2026-04-07T07:16:28.451Z","repository":{"id":36955906,"uuid":"466909944","full_name":"k1LoW/runn","owner":"k1LoW","description":"runn is a package/tool for running operations following a scenario.","archived":false,"fork":false,"pushed_at":"2024-10-29T14:32:59.000Z","size":4711,"stargazers_count":432,"open_issues_count":22,"forks_count":32,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-10-29T16:02:53.930Z","etag":null,"topics":["api-testing","automation","buf","buf-schema-registry","chrome-devtools-protocol","db-client","golang","grpc-client","hacktoberfest","http-client","load-testing","scenario-testing","ssh-client","test-helper","testing"],"latest_commit_sha":null,"homepage":"https://runn.run","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/k1LoW.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"k1LoW","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2022-03-07T02:01:45.000Z","updated_at":"2024-10-29T13:21:05.000Z","dependencies_parsed_at":"2023-07-13T04:16:34.767Z","dependency_job_id":"693c2244-7c8b-4ea2-a589-327ebbd9c29e","html_url":"https://github.com/k1LoW/runn","commit_stats":{"total_commits":2307,"total_committers":23,"mean_commits":"100.30434782608695","dds":0.3485045513654096,"last_synced_commit":"1e32358f2c179b1058944cabc248c1cb011e4dcf"},"previous_names":["k1low/runbk"],"tags_count":277,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k1LoW%2Frunn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k1LoW%2Frunn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k1LoW%2Frunn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k1LoW%2Frunn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/k1LoW","download_url":"https://codeload.github.com/k1LoW/runn/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247730069,"owners_count":20986404,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-testing","automation","buf","buf-schema-registry","chrome-devtools-protocol","db-client","golang","grpc-client","hacktoberfest","http-client","load-testing","scenario-testing","ssh-client","test-helper","testing"],"created_at":"2024-10-01T16:14:04.490Z","updated_at":"2026-04-01T18:10:42.762Z","avatar_url":"https://github.com/k1LoW.png","language":"Go","funding_links":["https://github.com/sponsors/k1LoW"],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/k1LoW/runn/raw/main/docs/logo.svg\" width=\"200\" alt=\"runn\"\u003e\n\u003c/p\u003e\n\n[![build](https://github.com/k1LoW/runn/actions/workflows/ci.yml/badge.svg)](https://github.com/k1LoW/runn/actions/workflows/ci.yml) ![Coverage](https://raw.githubusercontent.com/k1LoW/octocovs/main/badges/k1LoW/runn/coverage.svg) ![Code to Test Ratio](https://raw.githubusercontent.com/k1LoW/octocovs/main/badges/k1LoW/runn/ratio.svg) ![Test Execution Time](https://raw.githubusercontent.com/k1LoW/octocovs/main/badges/k1LoW/runn/time.svg) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/k1LoW/runn)\n\n`runn` ( means \"Run N\". is pronounced /rʌ́n én/. ) is a package/tool for running operations following a scenario.\n\nKey features of `runn` are:\n\n- **As a tool for scenario based testing.**\n- **As a test helper package for the Go language.**\n- **As a tool for workflow automation.**\n- **Support HTTP request, gRPC request, DB query, Chrome DevTools Protocol, and SSH/Local command execution**\n- **OpenAPI Document-like syntax for HTTP request testing.**\n- **Single binary = CI-Friendly.**\n\n## Online book\n\n- [runn Tutorial (Japanese)](https://zenn.dev/katzumi/books/runn-tutorial)\n- [runn cookbook (Japanese)](https://zenn.dev/k1low/books/runn-cookbook)\n\n## Quickstart\n\nYou can use the `runn new` command to quickly start creating scenarios ([runbooks](#runbook--runn-scenario-file-)).\n\n**:rocket: Create and run scenario using `curl` or `grpcurl` commands:**\n\n![docs/runn.svg](docs/runn.svg)\n\n\u003cdetails\u003e\n\n\u003csummary\u003eCommand details\u003c/summary\u003e\n\n``` console\n$ curl https://httpbin.org/json -H \"accept: application/json\"\n{\n  \"slideshow\": {\n    \"author\": \"Yours Truly\",\n    \"date\": \"date of publication\",\n    \"slides\": [\n      {\n        \"title\": \"Wake up to WonderWidgets!\",\n        \"type\": \"all\"\n      },\n      {\n        \"items\": [\n          \"Why \u003cem\u003eWonderWidgets\u003c/em\u003e are great\",\n          \"Who \u003cem\u003ebuys\u003c/em\u003e WonderWidgets\"\n        ],\n        \"title\": \"Overview\",\n        \"type\": \"all\"\n      }\n    ],\n    \"title\": \"Sample Slide Show\"\n  }\n}\n$ runn new --and-run --desc 'httpbin.org GET' --out http.yml -- curl https://httpbin.org/json -H \"accept: application/json\"\n$ grpcurl -d '{\"greeting\": \"alice\"}' grpcb.in:9001 hello.HelloService/SayHello\n{\n  \"reply\": \"hello alice\"\n}\n$ runn new --and-run --desc 'grpcb.in Call' --out grpc.yml -- grpcurl -d '{\"greeting\": \"alice\"}' grpcb.in:9001 hello.HelloService/SayHello\n$ runn list *.yml\n  Desc             Path      If\n---------------------------------\n  grpcb.in Call    grpc.yml\n  httpbin.org GET  http.yml\n$ runn run *.yml\n..\n\n2 scenarios, 0 skipped, 0 failures\n```\n\n\u003c/details\u003e\n\n**:rocket: Create scenario using access log:**\n\n![docs/runn_axslog.svg](docs/runn_axslog.svg)\n\n\u003cdetails\u003e\n\n\u003csummary\u003eCommand details\u003c/summary\u003e\n\n``` console\n$ cat access_log\n183.87.255.54 - - [18/May/2019:05:37:09 +0200] \"GET /?post=%3script%3ealert(1); HTTP/1.0\" 200 42433\n62.109.16.162 - - [18/May/2019:05:37:12 +0200] \"GET /core/files/js/editor.js/?form=\\xeb\\x2a\\x5e\\x89\\x76\\x08\\xc6\\x46\\x07\\x00\\xc7\\x46\\x0c\\x00\\x00\\x00\\x80\\xe8\\xdc\\xff\\xff\\xff/bin/sh HTTP/1.0\" 200 81956\n87.251.81.179 - - [18/May/2019:05:37:13 +0200] \"GET /login.php/?user=admin\u0026amount=100000 HTTP/1.0\" 400 4797\n103.36.79.144 - - [18/May/2019:05:37:14 +0200] \"GET /authorize.php/.well-known/assetlinks.json HTTP/1.0\" 200 9436\n$ cat access_log| runn new --out axslog.yml\n$ cat axslog.yml| yq\ndesc: Generated by `runn new`\nrunners:\n  req: https://dummy.example.com\nsteps:\n  - req:\n      /?post=%3script%3ealert(1);:\n        get:\n          body: null\n  - req:\n      /core/files/js/editor.js/?form=xebx2ax5ex89x76x08xc6x46x07x00xc7x46x0cx00x00x00x80xe8xdcxffxffxff/bin/sh:\n        get:\n          body: null\n  - req:\n      /login.php/?user=admin\u0026amount=100000:\n        get:\n          body: null\n  - req:\n      /authorize.php/.well-known/assetlinks.json:\n        get:\n          body: null\n$\n```\n\n\u003c/details\u003e\n\n## Usage\n\n`runn` can run a multi-step scenario following a `runbook` written in YAML format.\n\n### As a tool for scenario based testing / As a tool for automation.\n\n`runn` can run one or more runbooks as a CLI tool.\n\n``` console\n$ runn list path/to/**/*.yml\n  id:      desc:             if:       steps:  path\n-------------------------------------------------------------------------\n  a1b7b02  Only if included  included       2  p/t/only_if_included.yml\n  85ccd5f  List projects.                   4  p/t/p/list.yml\n  47d7ef7  List users.                      3  p/t/u/list.yml\n  97f9884  Login                            2  p/t/u/login.yml\n  2249d1b  Logout                           3  p/t/u/logout.yml\n$ runn run path/to/**/*.yml\nS....\n\n5 scenarios, 1 skipped, 0 failures\n```\n\n### As a test helper package for the Go language.\n\n`runn` can also behave as a test helper for the Go language.\n\n#### Run N runbooks using [httptest.Server](https://pkg.go.dev/net/http/httptest#Server) and [sql.DB](https://pkg.go.dev/database/sql#DB)\n\n``` go\nfunc TestRouter(t *testing.T) {\n\tctx := context.Background()\n\tdsn := \"username:password@tcp(localhost:3306)/testdb\"\n\tdb, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdbr, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tts := httptest.NewServer(NewRouter(db))\n\tt.Cleanup(func() {\n\t\tts.Close()\n\t\tdb.Close()\n\t\tdbr.Close()\n\t})\n\topts := []runn.Option{\n\t\trunn.T(t),\n\t\trunn.Runner(\"req\", ts.URL),\n\t\trunn.DBRunner(\"db\", dbr),\n\t}\n\to, err := runn.Load(\"testdata/books/**/*.yml\", opts...)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := o.RunN(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n```\n\n#### Run single runbook using [httptest.Server](https://pkg.go.dev/net/http/httptest#Server) and [sql.DB](https://pkg.go.dev/database/sql#DB)\n\n``` go\nfunc TestRouter(t *testing.T) {\n\tctx := context.Background()\n\tdsn := \"username:password@tcp(localhost:3306)/testdb\"\n\tdb, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdbr, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tts := httptest.NewServer(NewRouter(db))\n\tt.Cleanup(func() {\n\t\tts.Close()\n\t\tdb.Close()\n\t\tdbr.Close()\n\t})\n\topts := []runn.Option{\n\t\trunn.T(t),\n\t\trunn.Book(\"testdata/books/login.yml\"),\n\t\trunn.Runner(\"req\", ts.URL),\n\t\trunn.DBRunner(\"db\", dbr),\n\t}\n\to, err := runn.New(opts...)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := o.Run(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n```\n\n#### Run N runbooks using [grpc.Server](https://pkg.go.dev/google.golang.org/grpc#Server)\n\n``` go\nfunc TestServer(t *testing.T) {\n\taddr := \"127.0.0.1:8080\"\n\tl, err := net.Listen(\"tcp\", addr)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tts := grpc.NewServer()\n\tmyapppb.RegisterMyappServiceServer(s, NewMyappServer())\n\treflection.Register(s)\n\tgo func() {\n\t\tts.Serve(l)\n\t}()\n\tt.Cleanup(func() {\n\t\tts.GracefulStop()\n\t})\n\topts := []runn.Option{\n\t\trunn.T(t),\n\t\trunn.Runner(\"greq\", fmt.Sprintf(\"grpc://%s\", addr),\n\t}\n\to, err := runn.Load(\"testdata/books/**/*.yml\", opts...)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := o.RunN(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n```\n\n#### Run N runbooks with [http.Handler](https://pkg.go.dev/net/http#Handler) and [sql.DB](https://pkg.go.dev/database/sql#DB)\n\n``` go\nfunc TestRouter(t *testing.T) {\n\tctx := context.Background()\n\tdsn := \"username:password@tcp(localhost:3306)/testdb\"\n\tdb, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdbr, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tt.Cleanup(func() {\n\t\tdb.Close()\n\t\tdbr.Close()\n\t})\n\topts := []runn.Option{\n\t\trunn.T(t),\n\t\trunn.HTTPRunnerWithHandler(\"req\", NewRouter(db)),\n\t\trunn.DBRunner(\"db\", dbr),\n\t}\n\to, err := runn.Load(\"testdata/books/**/*.yml\", opts...)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := o.RunN(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n```\n\n## Examples\n\nSee the [details](./examples)\n\n## Runbook ( runn scenario file )\n\nThe runbook file has the following format.\n\n`step:` section accepts **list** or **ordered map**.\n\n**List:**\n\n``` yaml\ndesc: Login and get projects.\nrunners:\n  req: https://example.com/api/v1\n  db: mysql://root:mypass@localhost:3306/testdb\nvars:\n  username: alice\n  password: ${TEST_PASS}\nsteps:\n  -\n    db:\n      query: SELECT * FROM users WHERE name = '{{ vars.username }}'\n  -\n    req:\n      /login:\n        post:\n          body:\n            application/json:\n              email: \"{{ steps[0].rows[0].email }}\"\n              password: \"{{ vars.password }}\"\n    test: steps[1].res.status == 200\n  -\n    req:\n      /projects:\n        get:\n          headers:\n            Authorization: \"token {{ steps[1].res.body.session_token }}\"\n          body: null\n    test: steps[2].res.status == 200\n  -\n    test: len(steps[2].res.body.projects) \u003e 0\n```\n\n**Map:**\n\n``` yaml\ndesc: Login and get projects.\nrunners:\n  req: https://example.com/api/v1\n  db: mysql://root:mypass@localhost:3306/testdb\nvars:\n  username: alice\n  password: ${TEST_PASS}\nsteps:\n  find_user:\n    db:\n      query: SELECT * FROM users WHERE name = '{{ vars.username }}'\n  login:\n    req:\n      /login:\n        post:\n          body:\n            application/json:\n              email: \"{{ steps.find_user.rows[0].email }}\"\n              password: \"{{ vars.password }}\"\n    test: steps.login.res.status == 200\n  list_projects:\n    req:\n      /projects:\n        get:\n          headers:\n            Authorization: \"token {{ steps.login.res.body.session_token }}\"\n          body: null\n    test: steps.list_projects.res.status == 200\n  count_projects:\n    test: len(steps.list_projects.res.body.projects) \u003e 0\n```\n\n#### Grouping of related parts by color\n\n**List:**\n\n![color](docs/runbook.svg)\n\n**Map:**\n\n![color](docs/runbook_map.svg)\n\n### JSON Schema\n\nA JSON Schema for the runbook YAML format is available at [`runbook.schema.yaml`](runbook.schema.yaml).\n\nYou can use it with [YAML Language Server](https://github.com/redhat-developer/yaml-language-server) for editor validation and autocompletion by adding the following comment to the top of your runbook:\n\n``` yaml\n# yaml-language-server: $schema=https://raw.githubusercontent.com/k1LoW/runn/main/runbook.schema.yaml\n```\n\n### `desc:`\n\nDescription of runbook.\n\n``` yaml\ndesc: Login and get projects.\nrunners:\n  req: https://example.com/api/v1\nvars:\n  username: alice\nsteps:\n[...]\n```\n\n### `labels:`\n\nLabels of runbook.\n\n``` yaml\ndesc: Login\nrunners:\n  req: https://example.com/api/v1\nlabels:\n  - users\n  - auth\nsteps:\n[...]\n```\n\nRunbooks to be run can be filtered by labels.\n\n``` console\n$ runn run path/to/**/*.yml --label users --label projects\n```\n\n``` console\n$ runn run path/to/**/*.yml --label 'users and auth'\n```\n\n### `runners:`\n\nMapping of runners that run `steps:` of runbook.\n\nIn the `steps:` section, call the runner with the key specified in the `runners:` section.\n\nBuilt-in runners such as test runner do not need to be specified in this section.\n\n``` yaml\nrunners:\n  ghapi: ${GITHUB_API_ENDPOINT}\n  idp: https://auth.example.com\n  db: my:dbuser:${DB_PASS}@hostname:3306/dbname\n```\n\nIn the example, each runner can be called by `ghapi:`, `idp:` or `db:` in `steps:`.\n\n### `hostRules:`\n\nAllows remapping any request hostname to another hostname, IP address in HTTP/gRPC/DB/CDP/SSH runners.\n\n``` yaml\nhostRules:\n  example.com: 127.0.0.1:8080\n  '*.example.test': 192.168.0.16\n```\n\n### `vars:`\n\nMapping of variables available in the `steps:` of runbook.\n\n``` yaml\nvars:\n  username: alice@example.com\n  token: ${SECRET_TOKEN}\n```\n\nIn the example, each variable can be used in `{{ vars.username }}` or `{{ vars.token }}` in `steps:`.\n\n### `secrets:`\n\nList of secret var names to be masked.\n\n``` yaml\nsecrets:\n  - vars.secret_token\n  - binded_password\n  - current.res.message.token\n```\n\n### `debug:`\n\nEnable debug output for runn.\n\n``` yaml\ndebug: true\n```\n\n### `interval:`\n\nInterval between steps.\n\n```yaml\ninterval: 1\n```\n\n### `if:`\n\nConditions for skip all steps.\n\n``` yaml\nif: included # Run steps only if included\n```\n\n### `skipTest:`\n\nSkip all `test:` sections\n\n``` yaml\nskipTest: true\n```\n\n### `force:`\n\nForce all steps to run.\n\n``` yaml\nforce: true\n```\n\n### `trace:`\n\nAdd tokens for tracing to headers and queries by default.\n\nCurrently, HTTP runner, gRPC runner and DB runner are supported.\n\n``` yaml\ntrace: true\n```\n\n### `loop:`\n\nLoop setting for runbook.\n\n#### Simple loop runbook\n\n``` yaml\nloop: 10\nsteps:\n  [...]\n```\n\nor\n\n``` yaml\nloop:\n  count: 10\nsteps:\n  [...]\n```\n\n#### Retry runbook\n\nIt can be used as a retry mechanism by setting a condition in the `until:` section.\n\nIf the condition of `until:` is met, the loop is broken without waiting for the number of `count:` to be run.\n\nAlso, if the run of the number of `count:` completes but does not satisfy the condition of `until:`, then the step is considered to be failed.\n\n``` yaml\nloop:\n  count: 10\n  until: 'outcome == \"success\"' # until the runbook outcome is successful.\n  minInterval: 0.5 # sec\n  maxInterval: 10  # sec\n  # jitter: 0.0\n  # interval: 5\n  # multiplier: 1.5\nsteps:\n  waitingroom:\n    req:\n      /cart/in:\n        post:\n          body:\n[...]\n```\n\n- `outcome` ... the result of a completed (`success`, `failure`, `skipped`).\n\n### `concurrency:`\n\nRunbooks with the same key are assured of a single run at the same time.\n\n``` yaml\nconcurrency: use-shared-db\n```\n\nor\n\n``` yaml\nconcurrency:\n  - use-shared-db\n  - use-shared-api\n```\n\n### `needs:`\n\nIt is possible to identify runbooks that must be pre-run.\n\n``` yaml\nneeds:\n  prebook: path/to/prebook.yml\n  prebook2: path/to/prebook2.yml\n```\n\nValues bound by the bind runner can be referenced by `needs.\u003ckey\u003e. *`.\n\n### `steps:`\n\nSteps to run in runbook.\n\nThe steps are invoked in order from top to bottom.\n\nAny return values are recorded for each step.\n\nWhen `steps:` is array, recorded values can be retrieved with `{{ steps[*].* }}`.\n\n``` yaml\nsteps:\n  -\n    db:\n      query: SELECT * FROM users WHERE name = '{{ vars.username }}'\n  -\n    req:\n      /users/{{ steps[0].rows[0].id }}:\n        get:\n          body: null\n```\n\nWhen `steps:` is map, recorded values can be retrieved with `{{ steps.\u003ckey\u003e.* }}`.\n\n``` yaml\nsteps:\n  find_user:\n    db:\n      query: SELECT * FROM users WHERE name = '{{ vars.username }}'\n  user_info:\n    req:\n      /users/{{ steps.find_user.rows[0].id }}:\n        get:\n          body: null\n```\n\n### `steps[*].desc:` `steps.\u003ckey\u003e.desc:`\n\nDescription of step.\n\n``` yaml\nsteps:\n  -\n    desc: Login\n    req:\n      /login:\n        post:\n          body:\n[...]\n```\n\n### `steps[*].if:` `steps.\u003ckey\u003e.if:`\n\nConditions for skip step.\n\n``` yaml\nsteps:\n  login:\n    if: 'len(vars.token) == 0' # Run step only if var.token is not set\n    req:\n      /login:\n        post:\n          body:\n[...]\n```\n\n### `steps[*].loop:` `steps.\u003ckey\u003e.loop:`\n\nLoop setting for step.\n\n#### Simple loop step\n\n``` yaml\nsteps:\n  multicartin:\n    loop: 10\n    req:\n      /cart/in:\n        post:\n          body:\n            application/json:\n              product_id: \"{{ i }}\" # The loop count (0..9) is assigned to `i`.\n[...]\n```\n\nor\n\n``` yaml\nsteps:\n  multicartin:\n    loop:\n      count: 10\n    req:\n      /cart/in:\n        post:\n          body:\n            application/json:\n              product_id: \"{{ i }}\" # The loop count (0..9) is assigned to `i`.\n[...]\n```\n\n#### Retry step\n\nIt can be used as a retry mechanism by setting a condition in the `until:` section.\n\nIf the condition of `until:` is met, the loop is broken without waiting for the number of `count:` to be run.\n\nAlso, if the run of the number of `count:` completes but does not satisfy the condition of `until:`, then the step is considered to be failed.\n\n``` yaml\nsteps:\n  waitingroom:\n    loop:\n      count: 10\n      until: 'steps.waitingroom.res.status == \"201\"' # Store values of latest loop\n      minInterval: 500ms\n      maxInterval: 10 # sec\n      # jitter: 0.0\n      # interval: 5\n      # multiplier: 1.5\n    req:\n      /cart/in:\n        post:\n          body:\n[...]\n```\n\n#### Execution order within a loop\n\nWhen using `loop:`, it's important to understand the execution order of runners within each iteration:\n\n1. **Main runner** (HTTP, DB, gRPC, CDP, SSH, Exec, Include, or Runner) - Only one is executed\n2. **Dump runner** - Records request/response data (if specified)\n3. **Bind runner** - Binds variables (if specified)\n4. **Test runner** - Performs assertions (if specified)\n\nThis sequence repeats for each loop iteration.\n\n#### ⚠️ Warning: Avoid using `test:` with `loop.until:`\n\nWhen using `loop:` with an `until:` condition, **avoid** also using `test:` in the same step. The `test:` assertion runs on **every loop iteration**, which can cause the step to fail on the first iteration before the retry logic has a chance to work.\n\n**❌ Incorrect usage:**\n```yaml\nsteps:\n  retry_until_success:\n    req:\n      /status:\n        get:\n    loop:\n      count: 10\n      until: 'current.res.status == 200'  # Retry until successful\n    test: 'current.res.status == 200'     # This runs every iteration and may fail early!\n```\n\n**✅ Correct usage:**\n```yaml\nsteps:\n  retry_until_success:\n    req:\n      /status:\n        get:\n    loop:\n      count: 10\n      until: 'current.res.status == 200'  # This is sufficient for retry logic\n```\n\nThe `until:` condition already serves as your test condition for retry scenarios.\n\n### `steps[*].defer:` `steps.\u003ckey\u003e.defer:` [THIS IS EXPERIMENT]\n\nDeferring setting for step.\n\n```yaml\nsteps:\n  -\n    defer: true\n    req:\n      /cart:\n        delete:\n          body: null\n[...]\n```\n\nThe step marked `defer` behaves as follows.\n\n- If `defer: true` is set, run of the step is deferred until finish of the runbook.\n- Steps marked with `defer` are always run even if the running of intermediate steps fails.\n- If there are multiple steps marked with `defer`, they are run in LIFO order.\n    - Also, the included steps are added to run sequence of the parent runbook's deferred steps.\n\n### `steps[*].force:` `steps.\u003ckey\u003e.force:`\n\nForce step to run.\n\n```yaml\nsteps:\n[...]\n  -\n     force: true\n     dump: previous.res.body\n[...]\n```\n\n## Variables to be stored\n\nrunn can use variables and functions when running step.\n\nAlso, after step runs, HTTP responses, DB query results, etc. are automatically stored in variables.\n\nThe values are stored in predefined variables.\n\n| Variable name | Description |\n| --- | --- |\n| `vars` | Values set in the `vars:` section |\n| `steps` | Return values for each step |\n| `i` | Loop index (only in `loop:` section) |\n| `env` | Environment variables |\n| `current` | Return values of current step |\n| `previous` | Return values of previous step |\n| `parent` | Variables of parent runbook (only included) |\n\n## Variable Expansion\n\nrunn uses `{{ }}` syntax for variable expansion.\n\n### Object variable expansion\n\nWhen an object variable is used directly in `headers:` or `body:`, runn expands it as a map structure, not as a JSON string.\n\n``` yaml\nvars:\n  auth_headers:\n    X-Token: xxx\n    X-Api-Key: yyy\nsteps:\n  - req:\n      /api:\n        get:\n          headers: \"{{vars.auth_headers}}\"\n```\n\nIn the example, `vars.auth_headers` expands to multiple headers.\n\n### Converting object to JSON string\n\nTo convert an object to a JSON string, wrap the entire `{{...}}` template expression in single quotes. This signals `runn` to serialize the object to JSON.\n\n``` yaml\nvars:\n  metadata:\n    user: alice\n    role: admin\nsteps:\n  - req:\n      /api:\n        get:\n          headers:\n            X-Metadata: \"'{{vars.metadata}}'\"\n\n## Runner\n\n### HTTP Runner: Do HTTP request\n\nUse `https://` or `http://` scheme to specify HTTP Runner.\n\nWhen the step is invoked, it sends the specified HTTP Request and records the response.\n\n``` yaml\nrunners:\n  req: https://example.com\nsteps:\n  -\n    desc: Post /users                     # description of step\n    req:                                  # key to identify the runner. In this case, it is HTTP Runner.\n      /users:                             # path of http request\n        post:                             # method of http request\n          headers:                        # headers of http request\n            Authorization: 'Bearer xxxxx'\n          body:                           # body of http request\n            application/json:             # Content-Type specification. In this case, it is \"Content-Type: application/json\"\n              username: alice\n              password: passw0rd\n          trace: false                    # add `X-Runn-Trace` header to HTTP request for tracing\n    test: |                               # test for current step\n      current.res.status == 201\n```\n\nSee [testdata/book/http.yml](testdata/book/http.yml) and [testdata/book/http_multipart.yml](testdata/book/http_multipart.yml).\n\n#### Structure of recorded responses\n\nThe following response\n\n```\nHTTP/1.1 200 OK\nContent-Length: 29\nContent-Type: application/json\nDate: Wed, 07 Sep 2022 06:28:20 GMT\nSet-Cookie: cookie-name=cookie-value\n\n{\"data\":{\"username\":\"alice\"}}\n```\n\nis recorded with the following structure.\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  res:\n    status: 200                              # current.res.status\n    headers:\n      Content-Length:\n        - '29'                               # current.res.headers[\"Content-Length\"][0]\n      Content-Type:\n        - 'application/json'                 # current.res.headers[\"Content-Type\"][0]\n      Date:\n        - 'Wed, 07 Sep 2022 06:28:20 GMT'    # current.res.headers[\"Date\"][0]\n      Set-Cookie:\n        - 'cookie-name=cookie-value'         # current.res.headers[\"Set-Cookie\"][0]\n    cookies:\n      cookie-name: *http.Cookie              # current.res.cookies[\"cookie-name\"].Value\n    body:\n      data:\n        username: 'alice'                    # current.res.body.data.username\n    rawBody: '{\"data\":{\"username\":\"alice\"}}' # current.res.rawBody\n```\n\n#### Do not follow redirect\n\nThe HTTP Runner interprets HTTP responses and automatically redirects.\nTo disable this, set `notFollowRedirect` to true.\n\n``` yaml\nrunners:\n  req:\n    endpoint: https://example.com\n    notFollowRedirect: true\n```\n\n#### Enable Cookie Sending\n\nThe HTTP Runner automatically saves cookies by interpreting HTTP responses.\nTo enable cookie sending during requests, set `useCookie` to true.\n\n``` yaml\nrunners:\n  req:\n    endpoint: https://example.com\n    useCookie: true\n```\n\nSee [testdata/book/cookie.yml](testdata/book/cookie.yml) and [testdata/book/cookie_in_requests_automatically.yml](testdata/book/cookie_in_requests_automatically.yml).\n\n#### Validation of HTTP request and HTTP response\n\nHTTP requests sent by `runn` and their HTTP responses can be validated.\n\n**OpenAPI v3:**\n\n``` yaml\nrunners:\n  myapi:\n    endpoint: https://api.example.com\n    openapi3: path/to/openapi.yaml\n    # skipValidateRequest: false\n    # skipValidateResponse: false\n    # skipCircularReferenceCheck: false # skip checking circular references in OpenAPIv3 document.\n```\n\n#### Custom CA and Certificates\n\n``` yaml\nrunners:\n  myapi:\n    endpoint: https://api.github.com\n    cacert: path/to/cacert.pem\n    cert: path/to/cert.pem\n    key: path/to/key.pem\n    # skipVerify: false\n```\n\n#### Add `X-Runn-Trace` header to HTTP request for tracing\n\n``` yaml\nrunners:\n  myapi:\n    endpoint: https://api.github.com\n    trace: true\n```\n\n### gRPC Runner: Do gRPC request\n\nUse `grpc://` scheme to specify gRPC Runner.\n\nWhen the step is invoked, it sends the specified gRPC Request and records the response.\n\n``` yaml\nrunners:\n  greq: grpc://grpc.example.com:80\nsteps:\n  -\n    desc: Request using Unary RPC                     # description of step\n    greq:                                             # key to identify the runner. In this case, it is gRPC Runner.\n      grpctest.GrpcTestService/Hello:                 # package.Service/Method of rpc\n        headers:                                      # headers of rpc\n          authentication: tokenhello\n        message:                                      # message of rpc\n          name: alice\n          num: 3\n          request_time: 2022-06-25T05:24:43.861872Z\n        trace: false                                  # add `x-runn-trace` header to gRPC request for tracing\n  -\n    desc: Request using Server streaming RPC\n    greq:\n      grpctest.GrpcTestService/ListHello:\n        headers:\n          authentication: tokenlisthello\n        message:\n          name: bob\n          num: 4\n          request_time: 2022-06-25T05:24:43.861872Z\n        timeout: 3sec                                 # timeout for rpc\n    test: |\n      steps.server_streaming.res.status == 0 \u0026\u0026 len(steps.server_streaming.res.messages) \u003e 0\n  -\n    desc: Request using Client streaming RPC\n    greq:\n      grpctest.GrpcTestService/MultiHello:\n        headers:\n          authentication: tokenmultihello\n        messages:                                     # messages of rpc\n          -\n            name: alice\n            num: 5\n            request_time: 2022-06-25T05:24:43.861872Z\n          -\n            name: bob\n            num: 6\n            request_time: 2022-06-25T05:24:43.861872Z\n```\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    tls: true\n    cacert: path/to/cacert.pem\n    cert: path/to/cert.pem\n    key: path/to/key.pem\n    # skipVerify: false\n    # importPaths:\n    #   - protobuf/proto\n    # protos:\n    #   - general/health.proto\n    #   - myapp/**/*.proto\n```\n\nSee [testdata/book/grpc.yml](testdata/book/grpc.yml).\n\n#### Structure of recorded responses\n\nThe following response\n\n```protocol-buffer\nmessage HelloResponse {\n  string message = 1;\n\n  int32 num = 2;\n\n  google.protobuf.Timestamp create_time = 3;\n}\n```\n\n```json\n{\"create_time\":\"2022-06-25T05:24:43.861872Z\",\"message\":\"hello\",\"num\":32}\n```\n\n\nand headers\n\n```yaml\ncontent-type: [\"application/grpc\"]\nhello: [\"this is header\"]\n```\n\nand trailers\n\n```yaml\nhello: [\"this is trailer\"]\n```\n\nare recorded with the following structure.\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  res:\n    status: 0                                      # current.res.status\n    headers:\n      content-type:\n        - 'application/grpc'                       # current.res.headers[0].content-type\n      hello:\n        - 'this is header'                         # current.res.headers[0].hello\n    trailers:\n      hello:\n        - 'this is trailer'                        # current.res.trailers[0].hello\n    message:\n      create_time: '2022-06-25T05:24:43.861872Z'   # current.res.message.create_time\n      message: 'hello'                             # current.res.message.message\n      num: 32                                      # current.res.message.num\n    messages:\n      -\n        create_time: '2022-06-25T05:24:43.861872Z' # current.res.messages[0].create_time\n        message: 'hello'                           # current.res.messages[0].message\n        num: 32                                    # current.res.messages[0].num\n```\n\n#### Add `x-runn-trace` header to gRPC request for tracing\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    trace: true\n```\n\n#### Buf\n\ngRPC Runner supports Buf ecosystem includes [Buf Schema Registry](https://buf.build/product/bsr).\n\nIt can use the buf modules ( and protos ) it depends on.\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    bufDirs:\n      - path/to # Set buf directories for registering buf modules and protos\n```\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    bufLocks:\n      - path/to/buf.lock # Register buf modules using buf.lock\n```\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    bufConfigs:\n      - path/to/buf.yaml # Register buf modules using buf.yaml\n```\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    bufModules:\n        - buf.build/owner/repository\n        - buf.build/owner2/repository2\n```\n\n### DB Runner: Query a database\n\nUse dsn (Data Source Name) to specify DB Runner.\n\nWhen step is invoked, it executes the specified query the database.\n\n``` yaml\nrunners:\n  db: postgres://dbuser:dbpass@hostname:5432/dbname\nsteps:\n  -\n    desc: Select users            # description of step\n    db:                           # key to identify the runner. In this case, it is DB Runner.\n      query: SELECT * FROM users; # query to execute\n      trace: false                # add comment with trace token to query for tracing\n```\n\nSee [testdata/book/db.yml](testdata/book/db.yml).\n\n#### Structure of recorded responses\n\nIf the query is a SELECT clause, it records the selected `rows`,\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  rows:\n    -\n      id: 1                           # current.rows[0].id\n      username: 'alice'               # current.rows[0].username\n      password: 'passw0rd'            # current.rows[0].password\n      email: 'alice@example.com'      # current.rows[0].email\n      created: '2017-12-05T00:00:00Z' # current.rows[0].created\n    -\n      id: 2                           # current.rows[1].id\n      username: 'bob'                 # current.rows[1].username\n      password: 'passw0rd'            # current.rows[1].password\n      email: 'bob@example.com'        # current.rows[1].email\n      created: '2022-02-22T00:00:00Z' # current.rows[1].created\n```\n\notherwise it records `last_insert_id` and `rows_affected` .\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  last_insert_id: 3 # current.last_insert_id\n  rows_affected: 1  # current.rows_affected\n```\n\n#### Add comment with trace token to query for tracing\n\n``` yaml\nrunners:\n  db:\n    dsn: mysql://dbuser:dbpass@hostname:3306/dbname\n    trace: true\n```\n\n#### Support Databases\n\n**PostgreSQL:**\n\n``` yaml\nrunners:\n  mydb: postgres://dbuser:dbpass@hostname:5432/dbname\n```\n\n``` yaml\nrunners:\n  db: pg://dbuser:dbpass@hostname:5432/dbname\n```\n\n**MySQL:**\n\n``` yaml\nrunners:\n  testdb: mysql://dbuser:dbpass@hostname:3306/dbname\n```\n\n``` yaml\nrunners:\n  db: my://dbuser:dbpass@hostname:3306/dbname\n```\n\n**SQLite3:**\n\n``` yaml\nrunners:\n  db: sqlite:///path/to/dbname.db\n```\n\n``` yaml\nrunners:\n  local: sq://dbname.db\n```\n\n**Cloud Spanner:**\n\n``` yaml\nrunners:\n  testdb: spanner://test-project/test-instance/test-database\n```\n\n``` yaml\nrunners:\n  db: sp://test-project/test-instance/test-database\n```\n\n### CDP Runner: Control browser using Chrome DevTools Protocol (CDP)\n\nUse `cdp://` or `chrome://` scheme to specify CDP Runner.\n\nWhen the step is invoked, it controls browser via Chrome DevTools Protocol.\n\n``` yaml\nrunners:\n  cc: chrome://new\nsteps:\n  -\n    desc: Navigate, click and get h1 using CDP  # description of step\n    cc:                                         # key to identify the runner. In this case, it is CDP Runner.\n      actions:                                  # actions to control browser\n        - navigate: https://pkg.go.dev/time\n        - click: 'body \u003e header \u003e div.go-Header-inner \u003e nav \u003e div \u003e ul \u003e li:nth-child(2) \u003e a'\n        - waitVisible: 'body \u003e footer'\n        - text: 'h1'\n  -\n    test: |\n      previous.text == 'Install the latest version of Go'\n```\n\n#### CDP Configuration Options\n\nThe CDP runner supports additional configuration options:\n\n``` yaml\nrunners:\n  cc:\n    addr: chrome://new  # or cdp://new\n    timeout: 120sec     # Timeout for each CDP action (default: 60s)\n    flags:              # Chrome browser flags\n      headless: true\n      disable-gpu: true\n      no-sandbox: true\n```\n\n**Configuration parameters:**\n- `addr`: Chrome DevTools Protocol address. Use `chrome://new` or `cdp://new` to launch a new browser instance\n- `timeout`: Timeout duration for each CDP action/step (e.g., \"30s\", \"2m\", \"1m30s\"). Default is 60 seconds\n- `flags`: Chrome browser launch flags as key-value pairs\n\nSee [testdata/book/cdp.yml](testdata/book/cdp.yml).\n\n#### Functions for action to control browser\n\n\u003c!-- repin:fndoc --\u003e\n**`attributes`** (aliases: `getAttributes`, `attrs`, `getAttrs`)\n\nGet the element attributes for the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - attributes:\n      sel: \"h1\"\n# record to current.attrs:\n```\n\nor\n\n```yaml\nactions:\n  - attributes: \"h1\"\n```\n\n**`click`**\n\nSend a mouse click event to the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - click:\n      sel: \"nav \u003e div \u003e a\"\n```\n\nor\n\n```yaml\nactions:\n  - click: \"nav \u003e div \u003e a\"\n```\n\n**`doubleClick`**\n\nSend a mouse double click event to the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - doubleClick:\n      sel: \"nav \u003e div \u003e li\"\n```\n\nor\n\n```yaml\nactions:\n  - doubleClick: \"nav \u003e div \u003e li\"\n```\n\n**`evaluate`** (aliases: `eval`)\n\nEvaluate the Javascript expression (`expr`).\n\n```yaml\nactions:\n  - evaluate:\n      expr: \"document.querySelector(\\\"h1\\\").textContent = \\\"hello\\\"\"\n```\n\nor\n\n```yaml\nactions:\n  - evaluate: \"document.querySelector(\\\"h1\\\").textContent = \\\"hello\\\"\"\n```\n\n**`fullHTML`** (aliases: `getFullHTML`, `getHTML`, `html`)\n\nGet the full html of page.\n\n```yaml\nactions:\n  - fullHTML\n# record to current.html:\n```\n\n**`innerHTML`** (aliases: `getInnerHTML`)\n\nGet the inner html of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - innerHTML:\n      sel: \"h1\"\n# record to current.html:\n```\n\nor\n\n```yaml\nactions:\n  - innerHTML: \"h1\"\n```\n\n**`localStorage`** (aliases: `getLocalStorage`)\n\nGet localStorage items.\n\n```yaml\nactions:\n  - localStorage:\n      origin: \"https://github.com\"\n# record to current.items:\n```\n\nor\n\n```yaml\nactions:\n  - localStorage: \"https://github.com\"\n```\n\n**`location`** (aliases: `getLocation`)\n\nGet the document location.\n\n```yaml\nactions:\n  - location\n# record to current.url:\n```\n\n**`navigate`**\n\nNavigate the current frame to `url` page.\n\n```yaml\nactions:\n  - navigate:\n      url: \"https://pkg.go.dev/time\"\n```\n\nor\n\n```yaml\nactions:\n  - navigate: \"https://pkg.go.dev/time\"\n```\n\n**`outerHTML`** (aliases: `getOuterHTML`)\n\nGet the outer html of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - outerHTML:\n      sel: \"h1\"\n# record to current.html:\n```\n\nor\n\n```yaml\nactions:\n  - outerHTML: \"h1\"\n```\n\n**`screenshot`** (aliases: `getScreenshot`)\n\nTake a full screenshot of the entire browser viewport.\n\n```yaml\nactions:\n  - screenshot\n# record to current.png:\n```\n\n**`scroll`** (aliases: `scrollIntoView`)\n\nScroll the window to the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - scroll:\n      sel: \"body \u003e footer\"\n```\n\nor\n\n```yaml\nactions:\n  - scroll: \"body \u003e footer\"\n```\n\n**`sendKeys`**\n\nSend keys (`value`) to the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - sendKeys:\n      sel: \"input[name=username]\"\n      value: \"k1lowxb@gmail.com\"\n```\n\n**`sessionStorage`** (aliases: `getSessionStorage`)\n\nGet sessionStorage items.\n\n```yaml\nactions:\n  - sessionStorage:\n      origin: \"https://github.com\"\n# record to current.items:\n```\n\nor\n\n```yaml\nactions:\n  - sessionStorage: \"https://github.com\"\n```\n\n**`setUploadFile`** (aliases: `setUpload`)\n\nSet upload file (`path`) to the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - setUploadFile:\n      sel: \"input[name=avator]\"\n      path: \"/path/to/image.png\"\n```\n\n**`setUserAgent`** (aliases: `setUA`, `ua`, `userAgent`)\n\nSet the default User-Agent\n\n```yaml\nactions:\n  - setUserAgent:\n      userAgent: \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36\"\n```\n\nor\n\n```yaml\nactions:\n  - setUserAgent: \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36\"\n```\n\n**`submit`**\n\nSubmit the parent form of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - submit:\n      sel: \"form.login\"\n```\n\nor\n\n```yaml\nactions:\n  - submit: \"form.login\"\n```\n\n**`tabTo`**\n\nChange current frame to the tab with the specified `url`.\n\n```yaml\nactions:\n  - tabTo:\n      url: \"https://pkg.go.dev/time\"\n```\n\nor\n\n```yaml\nactions:\n  - tabTo: \"https://pkg.go.dev/time\"\n```\n\n**`text`** (aliases: `getText`)\n\nGet the visible text of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - text:\n      sel: \"h1\"\n# record to current.text:\n```\n\nor\n\n```yaml\nactions:\n  - text: \"h1\"\n```\n\n**`textContent`** (aliases: `getTextContent`)\n\nGet the text content of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - textContent:\n      sel: \"h1\"\n# record to current.text:\n```\n\nor\n\n```yaml\nactions:\n  - textContent: \"h1\"\n```\n\n**`title`** (aliases: `getTitle`)\n\nGet the document `title`.\n\n```yaml\nactions:\n  - title\n# record to current.title:\n```\n\n**`value`** (aliases: `getValue`)\n\nGet the Javascript value field of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - value:\n      sel: \"input[name=address]\"\n# record to current.value:\n```\n\nor\n\n```yaml\nactions:\n  - value: \"input[name=address]\"\n```\n\n**`wait`** (aliases: `sleep`)\n\nWait for the specified `time`.\n\n```yaml\nactions:\n  - wait:\n      time: \"10sec\"\n```\n\nor\n\n```yaml\nactions:\n  - wait: \"10sec\"\n```\n\n**`waitReady`**\n\nWait until the element matching the selector (`sel`) is ready.\n\n```yaml\nactions:\n  - waitReady:\n      sel: \"body \u003e footer\"\n```\n\nor\n\n```yaml\nactions:\n  - waitReady: \"body \u003e footer\"\n```\n\n**`waitVisible`**\n\nWait until the element matching the selector (`sel`) is visible.\n\n```yaml\nactions:\n  - waitVisible:\n      sel: \"body \u003e footer\"\n```\n\nor\n\n```yaml\nactions:\n  - waitVisible: \"body \u003e footer\"\n```\n\n\n\u003c!-- repin:fndoc --\u003e\n\n### SSH Runner: execute commands on a remote server connected via SSH\n\nUse `ssh://` scheme to specify SSH Runner.\n\nWhen step is invoked, it executes commands on a remote server connected via SSH.\n\n``` yaml\nrunners:\n  sc: ssh://username@hostname:port\nsteps:\n  -\n    desc: 'execute `hostname`' # description of step\n    sc:\n      command: hostname\n```\n\n\n``` yaml\nrunners:\n  sc:\n    hostname: hostname\n    user: username\n    port: 22\n    # host: myserver\n    # sshConfig: path/to/ssh_config\n    # keepSession: false\n    # localForward: '33306:127.0.0.1:3306'\n    # keyboardInteractive:\n    #   - match: Username\n    #     answer: k1low\n    #   - match: OTP\n    #     answer: ${MY_OTP}\n```\n\nSee [testdata/book/sshd.yml](testdata/book/sshd.yml).\n\n#### Structure of recorded responses\n\nThe response to the run command is always `stdout` and `stderr`.\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  stdout: 'hello world' # current.stdout\n  stderr: ''            # current.stderr\n```\n\n### Exec Runner: execute command\n\n\u003e **Note**\n\u003e Exec runner requires `run:exec` scope to run.\n\nThe `exec` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nIt executes command using `command:`, `stdin:`, `shell:`, `background:`, `liveOutput:` and `env:`.\n\n``` yaml\n-\n  exec:\n    command: grep hello\n    stdin: '{{ steps[3].res.rawBody }}'\n```\n\n``` yaml\n-\n  exec:\n    command: echo $0\n    shell: bash\n```\n\n`background:` set to `true` to run the command in the background.\n\n``` yaml\n-\n  exec:\n    command: kubectl port-forward svc/nginx 8080:80\n    liveOutput: true\n```\n\n`liveOutput:` set to `true` to output the command output live.\n\n``` yaml\n-\n  exec:\n    command: |\n      echo \"Start heavy commands\"\n      sleep 5\n      echo \"Heavy command finished\"\n    liveOutput: true\n```\n\n`env:` sets additional environment variables for the command execution.\n\n``` yaml\n-\n  exec:\n    command: printenv MY_VAR\n    env:\n      MY_VAR: hello\n      ANOTHER_VAR: \"{{ vars.value }}\"\n```\n\nSee [testdata/book/exec.yml](testdata/book/exec.yml).\n\n#### Structure of recorded responses\n\nThe response to the run command is always `stdout`, `stderr` and `exit_code`.\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  stdout: 'hello world' # current.stdout\n  stderr: ''            # current.stderr\n  exit_code: 0          # current.exit_code\n```\n\n#### `exec.shell:`\n\nUse `shell:` to define the shell and options to be used by the Exec runner.\n\n| Parameter| Command run internally |\n|:----|:----|\n| unspecified | `bash -e -c {0}` |\n| `bash` | `bash --noprofile --norc -eo pipefail -c {0}` |\n| `sh` | `sh -e -c {0}` |\n\n### Test Runner: test using recorded values\n\nThe `test` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nIt evaluates the conditional expression using the recorded values.\n\n``` yaml\n-\n  test: steps[3].res.status == 200\n```\n\nThe `test` runner can run in the same steps as the other runners.\n\n### Dump Runner: dump recorded values\n\nThe `dump` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nIt dumps the specified recorded values.\n\n``` yaml\n-\n  dump: steps[4].rows\n```\n\nor\n\n``` yaml\n-\n  dump:\n    expr: steps[4].rows\n    out: path/to/dump.out\n    disableTrailingNewline: true # disable trailing newline. default is false\n    disableMaskingSecrets: true  # disable masking secrets. default is false\n```\n\nThe `dump` runner can run in the same steps as the other runners.\n\n### Include Runner: include other runbook\n\nThe `include` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nInclude runner reads and runs the runbook in the specified path.\n\nRecorded values are nested.\n\n``` yaml\n-\n  include: path/to/get_token.yml\n```\n\nIt is also possible to override `vars:` of included runbook.\n\n``` yaml\n-\n  include:\n    path: path/to/login.yml\n    vars:\n      username: alice\n      password: alicepass\n-\n  include:\n    path: path/to/login.yml\n    vars:\n      username: bob\n      password: bobpass\n```\n\nIt is also possible to skip all `test:` sections in the included runbook.\n\n``` yaml\n-\n  include:\n    path: path/to/signup.yml\n    skipTest: true\n```\n\nIt is also possible to force all steps in the included runbook to run.\n\n``` yaml\n-\n  include:\n    path: path/to/signup.yml\n    force: true\n```\n\n### Bind Runner: bind variables\n\nThe `bind` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nIt bind runner binds any values with another key.\n\n``` yaml\n  -\n    req:\n      /users/k1low:\n        get:\n          body: null\n  -\n    bind:\n      user_id: steps[0].res.body.data.id\n  -\n    dump: user_id\n```\n\nThe `bind` runner can run in the same steps as the other runners.\n\n### Runner Runner: Define runner in the middle of steps.\n\nThe `runner` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nIt defines a runner in the middle of steps.\n\n``` yaml\n  -\n    runner:\n      sc: ssh://username@hostname:port\n  -\n    sc:\n      command: hostname\n```\n\nThe `runner` runner can not run in the same steps as the other runners.\n\n## Expression evaluation engine\n\nrunn has embedded [expr-lang/expr](https://github.com/expr-lang/expr) as the evaluation engine for the expression.\n\nSee [Language Definition](https://expr-lang.org/docs/language-definition).\n\n### Additional built-in functions\n\n- `urlencode` ... [url.QueryEscape](https://pkg.go.dev/net/url#QueryEscape)\n- `bool` ... [cast.ToBool](https://pkg.go.dev/github.com/spf13/cast#ToBool)\n- `compare` ... Compare two values ( `func(x, y any, ignorePaths ...string) bool` ). The optional `ignorePaths` argument is a list of [jq syntax path expressions](https://jqlang.github.io/jq/manual/#path) to ignore when comparing two values.\n- `diff` ... Difference between two values ( `func(x, y any, ignorePaths ...string) string` ). The optional `ignorePaths` argument is a list of [jq syntax path expressions](https://jqlang.github.io/jq/manual/#path) to ignore when comparing two values.\n- `pick` ... Returns same map type filtered by given keys left [lo.PickByKeys](https://github.com/samber/lo?tab=readme-ov-file#pickbykeys).\n- `omit` ... Returns same map type filtered by given keys excluded [lo.OmitByKeys](https://github.com/samber/lo?tab=readme-ov-file#omitbykeys).\n- `merge` ... Merges multiple maps from left to right [lo.Assign](https://github.com/samber/lo?tab=readme-ov-file#assign).\n- `input` ... [prompter.Prompt](https://pkg.go.dev/github.com/Songmu/prompter#Prompt)\n- `intersect` ... Find the intersection of two iterable values ( `func(x, y any) any` ).\n- `secret` ... [prompter.Password](https://pkg.go.dev/github.com/Songmu/prompter#Password)\n- `select` ... Select from candidates. `func(message string, candidates []string, default string) string`\n- `basename` ... [filepath.Base](https://pkg.go.dev/path/filepath#Base)\n- `time` ... Converts the given string or number to `time.Time{}`.\n- `faker.*` ... Generate fake data using [Faker](https://pkg.go.dev/github.com/k1LoW/runn/internal/builtin#Faker) ).\n- `file` ... Read the file as a string. Returns nil if it does not exist.\n- `hash.*` ... Compute hash values using secure algorithms. [hash.Sha256](https://pkg.go.dev/github.com/k1LoW/runn/internal/builtin#Hash.Sha256), [hash.Sha512](https://pkg.go.dev/github.com/k1LoW/runn/internal/builtin#Hash.Sha512).\n- `jwt.*` ... Generate and parse JSON Web Tokens (JWT) using the specified claims and signature algorithm. [jwt.Sign](https://pkg.go.dev/github.com/k1LoW/runn/internal/builtin#Jwt.Sign), [jwt.Parse](https://pkg.go.dev/github.com/k1LoW/runn/internal/builtin#Jwt.Parse).  \nSee [testdata/book/http_bearer.yml](https://github.com/k1LoW/runn/blob/main/testdata/book/http_bearer.yml) for a complete example.  \nNote: This function currently supports JWS (JSON Web Signature) only. JWE (JSON Web Encryption) is not supported.  \n\n\n## Option\n\nSee https://pkg.go.dev/github.com/k1LoW/runn#Option\n\n### Example: Run as a test helper ( func `T` )\n\nhttps://pkg.go.dev/github.com/k1LoW/runn#T\n\n``` go\no, err := runn.Load(\"testdata/**/*.yml\", runn.T(t))\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.RunN(ctx); err != nil {\n\tt.Fatal(err)\n}\n```\n\n### Example: Add custom function ( func `Func` )\n\nhttps://pkg.go.dev/github.com/k1LoW/runn#Func\n\n``` yaml\ndesc: Test using GitHub\nrunners:\n  req:\n    endpoint: https://github.com\nsteps:\n  -\n    req:\n      /search?l={{ urlencode('C++') }}\u0026q=runn\u0026type=Repositories:\n        get:\n          body:\n            application/json:\n              null\n    test: 'steps[0].res.status == 200'\n```\n\n``` go\no, err := runn.Load(\"testdata/**/*.yml\", runn.Func(\"urlencode\", url.QueryEscape))\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.RunN(ctx); err != nil {\n\tt.Fatal(err)\n}\n```\n\n## Scope\n\nrunn requires explicit specification of scope for some features.\n\nrunn has the following scopes.\n\n| Scope | Description | Default |\n| --- | --- | --- |\n| `read:parent` | Required for reading files above the working directory. | `false` |\n| `read:remote` | Required for reading remote files. | `false` |\n| `run:exec` | Required for running Exec runner. | `false` |\n\nTo specify scopes, using the `--scopes` option or the environment variable `RUNN_SCOPES`.\n\n```console\n$ runn run path/to/**/*.yml --scopes read:parent,read:remote\n```\n\n```console\n$ env RUNN_SCOPES=read:parent,read:remote runn run path/to/**/*.yml\n```\n\nAlso, [runn.Scopes](https://pkg.go.dev/github.com/k1LoW/runn#Scopes) can be used in the code\n\n``` go\no, err := runn.Load(\"path/to/**/*.yml\", runn.Scopes(runn.AllowReadParent, runn.AllowReadRemote))\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.RunN(ctx); err != nil {\n\tt.Fatal(err)\n}\n```\n\nTo disable scope, can use `!read:*` instead of `read:*`\n\n```console\n$ runn run path/to/**/*.yml --scopes '!read:parent'\n```\n\n## Filter runbooks to be executed by the environment variable `RUNN_RUN`\n\nRun only runbooks matching the filename \"login\".\n\n``` console\n$ env RUNN_RUN=login go test ./... -run TestRouter\n```\n\n## Measure elapsed time as profile\n\n``` go\nopts := []runn.Option{\n\trunn.T(t),\n\trunn.Book(\"testdata/books/login.yml\"),\n\trunn.Profile(true),\n}\no, err := runn.New(opts...)\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.Run(ctx); err != nil {\n\tt.Fatal(err)\n}\nf, err := os.Open(\"profile.json\")\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.DumpProfile(f); err != nil {\n\tt.Fatal(err)\n}\n```\n\nor\n\n``` console\n$ runn run testdata/books/login.yml --profile\n```\n\nThe runbook run profile can be read with `runn rprof` command.\n\n``` console\n$ runn rprof runn.prof\n  runbook[login site](t/b/login.yml)           2995.72ms\n    steps[0].req                                747.67ms\n    steps[1].req                                185.69ms\n    steps[2].req                                192.65ms\n    steps[3].req                                188.23ms\n    steps[4].req                                569.53ms\n    steps[5].req                                299.88ms\n    steps[6].test                                 0.14ms\n    steps[7].include                            620.88ms\n      runbook[include](t/b/login_include.yml)   605.56ms\n        steps[0].req                            605.54ms\n    steps[8].req                                190.92ms\n  [total]                                      2995.84ms\n```\n\n## Capture runbook runs\n\n``` go\nopts := []runn.Option{\n\trunn.T(t),\n\trunn.Capture(capture.Runbook(\"path/to/dir\")),\n}\no, err := runn.Load(\"testdata/books/**/*.yml\", opts...)\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.RunN(ctx); err != nil {\n\tt.Fatal(err)\n}\n```\n\nor\n\n``` console\n$ runn run path/to/**/*.yml --capture path/to/dir\n```\n\n## Load test using runbooks\n\nYou can use the `runn loadt` command for load testing using runbooks.\n\n``` console\n$ runn loadt --load-concurrent 2 --max-rps 0 path/to/*.yml\n\nNumber of runbooks per RunN....: 15\nWarm up time (--warm-up).......: 5s\nDuration (--duration)..........: 10s\nConcurrent (--load-concurrent).: 2\nMax RunN per second (--max-rps): 0\n\nTotal..........................: 12\nSucceeded......................: 12\nFailed.........................: 0\nError rate.....................: 0%\nRunN per seconds...............: 1.2\nLatency .......................: max=1,835.1ms min=1,451.3ms avg=1,627.8ms med=1,619.8ms p(90)=1,741.5ms p(99)=1,788.4ms\n\n```\n\nIt also checks the results of the load test with the `--threshold` option. If the condition is not met, it returns exit status 1.\n\n``` console\n$ runn loadt --load-concurrent 2 --max-rps 0 --threshold 'error_rate \u003c 10' path/to/*.yml\n\nNumber of runbooks per RunN...: 15\nWarm up time (--warm-up)......: 5s\nDuration (--duration).........: 10s\nConcurrent (--load-concurrent): 2\n\nTotal.........................: 13\nSucceeded.....................: 12\nFailed........................: 1\nError rate....................: 7.6%\nRunN per seconds..............: 1.3\nLatency ......................: max=1,790.2ms min=95.0ms avg=1,541.4ms med=1,640.4ms p(90)=1,749.7ms p(99)=1,786.5ms\n\nError: (error_rate \u003c 10) is not true\nerror_rate \u003c 10\n├── error_rate =\u003e 14.285714285714285\n└── 10 =\u003e 10\n```\n\n### Variables for threshold\n\n| Variable name | Type | Description |\n| --- | --- | --- |\n| `total` | `int` | Total |\n| `succeeded` | `int` | Succeeded |\n| `failed` | `int` | Failed |\n| `error_rate` | `float` | Error rate |\n| `rps` | `float` | RunN per seconds |\n| `max` | `float` | Latency max (ms) |\n| `mid` | `float` | Latency mid (ms) |\n| `min` | `float` | Latency min (ms) |\n| `p90` | `float` | Latency p(90) (ms) |\n| `p99` | `float` | Latency p(99) (ms) |\n| `avg` | `float` | Latency avg (ms) |\n\n## Install\n\n### As a CLI tool\n\n**deb:**\n\n``` console\n$ export RUNN_VERSION=X.X.X\n$ curl -o runn.deb -L https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.deb\n$ dpkg -i runn.deb\n```\n\n**RPM:**\n\n``` console\n$ export RUNN_VERSION=X.X.X\n$ yum install https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.rpm\n```\n\n**apk:**\n\n``` console\n$ export RUNN_VERSION=X.X.X\n$ curl -o runn.apk -L https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.apk\n$ apk add runn.apk\n```\n\n**homebrew tap:**\n\n```console\n$ brew install k1LoW/tap/runn\n```\n\n**[aqua](https://aquaproj.github.io/):**\n\n```console\n$ aqua g -i k1LoW/runn\n```\n\n**manually:**\n\nDownload binary from [releases page](https://github.com/k1LoW/runn/releases)\n\n**docker:**\n\n```console\n$ docker container run -it --rm --name runn -v $PWD:/books ghcr.io/k1low/runn:latest list /books/*.yml\n```\n\n**go install:**\n\n```console\n$ go install github.com/k1LoW/runn/cmd/runn@latest\n```\n\n### As a test helper\n\n```console\n$ go get github.com/k1LoW/runn\n```\n\n## Alternatives\n\n- [zoncoen/scenarigo](https://github.com/zoncoen/scenarigo): An end-to-end scenario testing tool for HTTP/gRPC server.\n\n## References\n\n- [zoncoen/scenarigo](https://github.com/zoncoen/scenarigo): An end-to-end scenario testing tool for HTTP/gRPC server.\n- [fullstorydev/grpcurl](https://github.com/fullstorydev/grpcurl): Like cURL, but for gRPC: Command-line tool for interacting with gRPC servers\n- [ktr0731/evans](https://github.com/ktr0731/evans): Evans: more expressive universal gRPC client\n\n## License\n\n- [MIT License](LICENSE)\n    - Include logo as well as source code.\n    - Only logo license can be selected [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).\n    - Also, if there is no alteration to the logo and it is used for technical information about runn, I would not say anything if the copyright notice is omitted.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk1low%2Frunn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fk1low%2Frunn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk1low%2Frunn/lists"}