{"id":13511251,"url":"https://github.com/k1LoW/runn","last_synced_at":"2025-03-30T20:32:50.610Z","repository":{"id":36955906,"uuid":"466909944","full_name":"k1LoW/runn","owner":"k1LoW","description":"runn is a package/tool for running operations following a scenario.","archived":false,"fork":false,"pushed_at":"2024-10-29T14:32:59.000Z","size":4711,"stargazers_count":432,"open_issues_count":22,"forks_count":32,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-10-29T16:02:53.930Z","etag":null,"topics":["api-testing","automation","buf","buf-schema-registry","chrome-devtools-protocol","db-client","golang","grpc-client","hacktoberfest","http-client","load-testing","scenario-testing","ssh-client","test-helper","testing"],"latest_commit_sha":null,"homepage":"https://runn.run","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/k1LoW.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"k1LoW","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2022-03-07T02:01:45.000Z","updated_at":"2024-10-29T13:21:05.000Z","dependencies_parsed_at":"2023-07-13T04:16:34.767Z","dependency_job_id":"693c2244-7c8b-4ea2-a589-327ebbd9c29e","html_url":"https://github.com/k1LoW/runn","commit_stats":null,"previous_names":["k1low/runbk"],"tags_count":244,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k1LoW%2Frunn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k1LoW%2Frunn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k1LoW%2Frunn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k1LoW%2Frunn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/k1LoW","download_url":"https://codeload.github.com/k1LoW/runn/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222581190,"owners_count":17006295,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-testing","automation","buf","buf-schema-registry","chrome-devtools-protocol","db-client","golang","grpc-client","hacktoberfest","http-client","load-testing","scenario-testing","ssh-client","test-helper","testing"],"created_at":"2024-08-01T03:00:41.820Z","updated_at":"2025-03-30T20:32:50.596Z","avatar_url":"https://github.com/k1LoW.png","language":"Go","readme":"\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/k1LoW/runn/raw/main/docs/logo.svg\" width=\"200\" alt=\"runn\"\u003e\n\u003c/p\u003e\n\n[![build](https://github.com/k1LoW/runn/actions/workflows/ci.yml/badge.svg)](https://github.com/k1LoW/runn/actions/workflows/ci.yml) ![Coverage](https://raw.githubusercontent.com/k1LoW/octocovs/main/badges/k1LoW/runn/coverage.svg) ![Code to Test Ratio](https://raw.githubusercontent.com/k1LoW/octocovs/main/badges/k1LoW/runn/ratio.svg) ![Test Execution Time](https://raw.githubusercontent.com/k1LoW/octocovs/main/badges/k1LoW/runn/time.svg)\n\n`runn` ( means \"Run N\". is pronounced /rʌ́n én/. ) is a package/tool for running operations following a scenario.\n\nKey features of `runn` are:\n\n- **As a tool for scenario based testing.**\n- **As a test helper package for the Go language.**\n- **As a tool for workflow automation.**\n- **Support HTTP request, gRPC request, DB query, Chrome DevTools Protocol, and SSH/Local command execution**\n- **OpenAPI Document-like syntax for HTTP request testing.**\n- **Single binary = CI-Friendly.**\n\n## Online book\n\n- [runn Tutorial (Japanese)](https://zenn.dev/katzumi/books/runn-tutorial)\n- [runn cookbook (Japanese)](https://zenn.dev/k1low/books/runn-cookbook)\n\n## Quickstart\n\nYou can use the `runn new` command to quickly start creating scenarios ([runbooks](#runbook--runn-scenario-file-)).\n\n**:rocket: Create and run scenario using `curl` or `grpcurl` commands:**\n\n![docs/runn.svg](docs/runn.svg)\n\n\u003cdetails\u003e\n\n\u003csummary\u003eCommand details\u003c/summary\u003e\n\n``` console\n$ curl https://httpbin.org/json -H \"accept: application/json\"\n{\n  \"slideshow\": {\n    \"author\": \"Yours Truly\",\n    \"date\": \"date of publication\",\n    \"slides\": [\n      {\n        \"title\": \"Wake up to WonderWidgets!\",\n        \"type\": \"all\"\n      },\n      {\n        \"items\": [\n          \"Why \u003cem\u003eWonderWidgets\u003c/em\u003e are great\",\n          \"Who \u003cem\u003ebuys\u003c/em\u003e WonderWidgets\"\n        ],\n        \"title\": \"Overview\",\n        \"type\": \"all\"\n      }\n    ],\n    \"title\": \"Sample Slide Show\"\n  }\n}\n$ runn new --and-run --desc 'httpbin.org GET' --out http.yml -- curl https://httpbin.org/json -H \"accept: application/json\"\n$ grpcurl -d '{\"greeting\": \"alice\"}' grpcb.in:9001 hello.HelloService/SayHello\n{\n  \"reply\": \"hello alice\"\n}\n$ runn new --and-run --desc 'grpcb.in Call' --out grpc.yml -- grpcurl -d '{\"greeting\": \"alice\"}' grpcb.in:9001 hello.HelloService/SayHello\n$ runn list *.yml\n  Desc             Path      If\n---------------------------------\n  grpcb.in Call    grpc.yml\n  httpbin.org GET  http.yml\n$ runn run *.yml\n..\n\n2 scenarios, 0 skipped, 0 failures\n```\n\n\u003c/details\u003e\n\n**:rocket: Create scenario using access log:**\n\n![docs/runn_axslog.svg](docs/runn_axslog.svg)\n\n\u003cdetails\u003e\n\n\u003csummary\u003eCommand details\u003c/summary\u003e\n\n``` console\n$ cat access_log\n183.87.255.54 - - [18/May/2019:05:37:09 +0200] \"GET /?post=%3script%3ealert(1); HTTP/1.0\" 200 42433\n62.109.16.162 - - [18/May/2019:05:37:12 +0200] \"GET /core/files/js/editor.js/?form=\\xeb\\x2a\\x5e\\x89\\x76\\x08\\xc6\\x46\\x07\\x00\\xc7\\x46\\x0c\\x00\\x00\\x00\\x80\\xe8\\xdc\\xff\\xff\\xff/bin/sh HTTP/1.0\" 200 81956\n87.251.81.179 - - [18/May/2019:05:37:13 +0200] \"GET /login.php/?user=admin\u0026amount=100000 HTTP/1.0\" 400 4797\n103.36.79.144 - - [18/May/2019:05:37:14 +0200] \"GET /authorize.php/.well-known/assetlinks.json HTTP/1.0\" 200 9436\n$ cat access_log| runn new --out axslog.yml\n$ cat axslog.yml| yq\ndesc: Generated by `runn new`\nrunners:\n  req: https://dummy.example.com\nsteps:\n  - req:\n      /?post=%3script%3ealert(1);:\n        get:\n          body: null\n  - req:\n      /core/files/js/editor.js/?form=xebx2ax5ex89x76x08xc6x46x07x00xc7x46x0cx00x00x00x80xe8xdcxffxffxff/bin/sh:\n        get:\n          body: null\n  - req:\n      /login.php/?user=admin\u0026amount=100000:\n        get:\n          body: null\n  - req:\n      /authorize.php/.well-known/assetlinks.json:\n        get:\n          body: null\n$\n```\n\n\u003c/details\u003e\n\n## Usage\n\n`runn` can run a multi-step scenario following a `runbook` written in YAML format.\n\n### As a tool for scenario based testing / As a tool for automation.\n\n`runn` can run one or more runbooks as a CLI tool.\n\n``` console\n$ runn list path/to/**/*.yml\n  id:      desc:             if:       steps:  path\n-------------------------------------------------------------------------\n  a1b7b02  Only if included  included       2  p/t/only_if_included.yml\n  85ccd5f  List projects.                   4  p/t/p/list.yml\n  47d7ef7  List users.                      3  p/t/u/list.yml\n  97f9884  Login                            2  p/t/u/login.yml\n  2249d1b  Logout                           3  p/t/u/logout.yml\n$ runn run path/to/**/*.yml\nS....\n\n5 scenarios, 1 skipped, 0 failures\n```\n\n### As a test helper package for the Go language.\n\n`runn` can also behave as a test helper for the Go language.\n\n#### Run N runbooks using [httptest.Server](https://pkg.go.dev/net/http/httptest#Server) and [sql.DB](https://pkg.go.dev/database/sql#DB)\n\n``` go\nfunc TestRouter(t *testing.T) {\n\tctx := context.Background()\n\tdsn := \"username:password@tcp(localhost:3306)/testdb\"\n\tdb, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdbr, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tts := httptest.NewServer(NewRouter(db))\n\tt.Cleanup(func() {\n\t\tts.Close()\n\t\tdb.Close()\n\t\tdbr.Close()\n\t})\n\topts := []runn.Option{\n\t\trunn.T(t),\n\t\trunn.Runner(\"req\", ts.URL),\n\t\trunn.DBRunner(\"db\", dbr),\n\t}\n\to, err := runn.Load(\"testdata/books/**/*.yml\", opts...)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := o.RunN(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n```\n\n#### Run single runbook using [httptest.Server](https://pkg.go.dev/net/http/httptest#Server) and [sql.DB](https://pkg.go.dev/database/sql#DB)\n\n``` go\nfunc TestRouter(t *testing.T) {\n\tctx := context.Background()\n\tdsn := \"username:password@tcp(localhost:3306)/testdb\"\n\tdb, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdbr, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tts := httptest.NewServer(NewRouter(db))\n\tt.Cleanup(func() {\n\t\tts.Close()\n\t\tdb.Close()\n\t\tdbr.Close()\n\t})\n\topts := []runn.Option{\n\t\trunn.T(t),\n\t\trunn.Book(\"testdata/books/login.yml\"),\n\t\trunn.Runner(\"req\", ts.URL),\n\t\trunn.DBRunner(\"db\", dbr),\n\t}\n\to, err := runn.New(opts...)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := o.Run(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n```\n\n#### Run N runbooks using [grpc.Server](https://pkg.go.dev/google.golang.org/grpc#Server)\n\n``` go\nfunc TestServer(t *testing.T) {\n\taddr := \"127.0.0.1:8080\"\n\tl, err := net.Listen(\"tcp\", addr)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tts := grpc.NewServer()\n\tmyapppb.RegisterMyappServiceServer(s, NewMyappServer())\n\treflection.Register(s)\n\tgo func() {\n\t\tts.Serve(l)\n\t}()\n\tt.Cleanup(func() {\n\t\tts.GracefulStop()\n\t})\n\topts := []runn.Option{\n\t\trunn.T(t),\n\t\trunn.Runner(\"greq\", fmt.Sprintf(\"grpc://%s\", addr),\n\t}\n\to, err := runn.Load(\"testdata/books/**/*.yml\", opts...)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := o.RunN(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n```\n\n#### Run N runbooks with [http.Handler](https://pkg.go.dev/net/http#Handler) and [sql.DB](https://pkg.go.dev/database/sql#DB)\n\n``` go\nfunc TestRouter(t *testing.T) {\n\tctx := context.Background()\n\tdsn := \"username:password@tcp(localhost:3306)/testdb\"\n\tdb, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdbr, err := sql.Open(\"mysql\", dsn)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tt.Cleanup(func() {\n\t\tdb.Close()\n\t\tdbr.Close()\n\t})\n\topts := []runn.Option{\n\t\trunn.T(t),\n\t\trunn.HTTPRunnerWithHandler(\"req\", NewRouter(db)),\n\t\trunn.DBRunner(\"db\", dbr),\n\t}\n\to, err := runn.Load(\"testdata/books/**/*.yml\", opts...)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := o.RunN(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n```\n\n## Examples\n\nSee the [details](./examples)\n\n## Runbook ( runn scenario file )\n\nThe runbook file has the following format.\n\n`step:` section accepts **list** or **ordered map**.\n\n**List:**\n\n``` yaml\ndesc: Login and get projects.\nrunners:\n  req: https://example.com/api/v1\n  db: mysql://root:mypass@localhost:3306/testdb\nvars:\n  username: alice\n  password: ${TEST_PASS}\nsteps:\n  -\n    db:\n      query: SELECT * FROM users WHERE name = '{{ vars.username }}'\n  -\n    req:\n      /login:\n        post:\n          body:\n            application/json:\n              email: \"{{ steps[0].rows[0].email }}\"\n              password: \"{{ vars.password }}\"\n    test: steps[1].res.status == 200\n  -\n    req:\n      /projects:\n        get:\n          headers:\n            Authorization: \"token {{ steps[1].res.body.session_token }}\"\n          body: null\n    test: steps[2].res.status == 200\n  -\n    test: len(steps[2].res.body.projects) \u003e 0\n```\n\n**Map:**\n\n``` yaml\ndesc: Login and get projects.\nrunners:\n  req: https://example.com/api/v1\n  db: mysql://root:mypass@localhost:3306/testdb\nvars:\n  username: alice\n  password: ${TEST_PASS}\nsteps:\n  find_user:\n    db:\n      query: SELECT * FROM users WHERE name = '{{ vars.username }}'\n  login:\n    req:\n      /login:\n        post:\n          body:\n            application/json:\n              email: \"{{ steps.find_user.rows[0].email }}\"\n              password: \"{{ vars.password }}\"\n    test: steps.login.res.status == 200\n  list_projects:\n    req:\n      /projects:\n        get:\n          headers:\n            Authorization: \"token {{ steps.login.res.body.session_token }}\"\n          body: null\n    test: steps.list_projects.res.status == 200\n  count_projects:\n    test: len(steps.list_projects.res.body.projects) \u003e 0\n```\n\n#### Grouping of related parts by color\n\n**List:**\n\n![color](docs/runbook.svg)\n\n**Map:**\n\n![color](docs/runbook_map.svg)\n\n### `desc:`\n\nDescription of runbook.\n\n``` yaml\ndesc: Login and get projects.\nrunners:\n  req: https://example.com/api/v1\nvars:\n  username: alice\nsteps:\n[...]\n```\n\n### `labels:`\n\nLabels of runbook.\n\n``` yaml\ndesc: Login\nrunners:\n  req: https://example.com/api/v1\nlabels:\n  - users\n  - auth\nsteps:\n[...]\n```\n\nRunbooks to be run can be filtered by labels.\n\n``` console\n$ runn run path/to/**/*.yml --label users --label projects\n```\n\n``` console\n$ runn run path/to/**/*.yml --label 'users and auth'\n```\n\n### `runners:`\n\nMapping of runners that run `steps:` of runbook.\n\nIn the `steps:` section, call the runner with the key specified in the `runners:` section.\n\nBuilt-in runners such as test runner do not need to be specified in this section.\n\n``` yaml\nrunners:\n  ghapi: ${GITHUB_API_ENDPOINT}\n  idp: https://auth.example.com\n  db: my:dbuser:${DB_PASS}@hostname:3306/dbname\n```\n\nIn the example, each runner can be called by `ghapi:`, `idp:` or `db:` in `steps:`.\n\n### `hostRules:`\n\nAllows remapping any request hostname to another hostname, IP address in HTTP/gRPC/DB/CDP/SSH runners.\n\n``` yaml\nhostRules:\n  example.com: 127.0.0.1:8080\n  '*.example.test': 192.168.0.16\n```\n\n### `vars:`\n\nMapping of variables available in the `steps:` of runbook.\n\n``` yaml\nvars:\n  username: alice@example.com\n  token: ${SECRET_TOKEN}\n```\n\nIn the example, each variable can be used in `{{ vars.username }}` or `{{ vars.token }}` in `steps:`.\n\n### `secrets:`\n\nList of secret var names to be masked.\n\n``` yaml\nsecrets:\n  - vars.secret_token\n  - binded_password\n  - current.res.message.token\n```\n\n### `debug:`\n\nEnable debug output for runn.\n\n``` yaml\ndebug: true\n```\n\n### `interval:`\n\nInterval between steps.\n\n```yaml\ninterval: 1\n```\n\n### `if:`\n\nConditions for skip all steps.\n\n``` yaml\nif: included # Run steps only if included\n```\n\n### `skipTest:`\n\nSkip all `test:` sections\n\n``` yaml\nskipTest: true\n```\n\n### `force:`\n\nForce all steps to run.\n\n``` yaml\nforce: true\n```\n\n### `trace:`\n\nAdd tokens for tracing to headers and queries by default.\n\nCurrently, HTTP runner, gRPC runner and DB runner are supported.\n\n``` yaml\ntrace: true\n```\n\n### `loop:`\n\nLoop setting for runbook.\n\n#### Simple loop runbook\n\n``` yaml\nloop: 10\nsteps:\n  [...]\n```\n\nor\n\n``` yaml\nloop:\n  count: 10\nsteps:\n  [...]\n```\n\n#### Retry runbook\n\nIt can be used as a retry mechanism by setting a condition in the `until:` section.\n\nIf the condition of `until:` is met, the loop is broken without waiting for the number of `count:` to be run.\n\nAlso, if the run of the number of `count:` completes but does not satisfy the condition of `until:`, then the step is considered to be failed.\n\n``` yaml\nloop:\n  count: 10\n  until: 'outcome == \"success\"' # until the runbook outcome is successful.\n  minInterval: 0.5 # sec\n  maxInterval: 10  # sec\n  # jitter: 0.0\n  # interval: 5\n  # multiplier: 1.5\nsteps:\n  waitingroom:\n    req:\n      /cart/in:\n        post:\n          body:\n[...]\n```\n\n- `outcome` ... the result of a completed (`success`, `failure`, `skipped`).\n\n### `concurrency:`\n\nRunbooks with the same key are assured of a single run at the same time.\n\n``` yaml\nconcurrency: use-shared-db\n```\n\nor\n\n``` yaml\nconcurrency:\n  - use-shared-db\n  - use-shared-api\n```\n\n### `needs:`\n\nIt is possible to identify runbooks that must be pre-run.\n\n``` yaml\nneeds:\n  prebook: path/to/prebook.yml\n  prebook2: path/to/prebook2.yml\n```\n\nValues bound by the bind runner can be referenced by `needs.\u003ckey\u003e. *`.\n\n### `steps:`\n\nSteps to run in runbook.\n\nThe steps are invoked in order from top to bottom.\n\nAny return values are recorded for each step.\n\nWhen `steps:` is array, recorded values can be retrieved with `{{ steps[*].* }}`.\n\n``` yaml\nsteps:\n  -\n    db:\n      query: SELECT * FROM users WHERE name = '{{ vars.username }}'\n  -\n    req:\n      /users/{{ steps[0].rows[0].id }}:\n        get:\n          body: null\n```\n\nWhen `steps:` is map, recorded values can be retrieved with `{{ steps.\u003ckey\u003e.* }}`.\n\n``` yaml\nsteps:\n  find_user:\n    db:\n      query: SELECT * FROM users WHERE name = '{{ vars.username }}'\n  user_info:\n    req:\n      /users/{{ steps.find_user.rows[0].id }}:\n        get:\n          body: null\n```\n\n### `steps[*].desc:` `steps.\u003ckey\u003e.desc:`\n\nDescription of step.\n\n``` yaml\nsteps:\n  -\n    desc: Login\n    req:\n      /login:\n        post:\n          body:\n[...]\n```\n\n### `steps[*].if:` `steps.\u003ckey\u003e.if:`\n\nConditions for skip step.\n\n``` yaml\nsteps:\n  login:\n    if: 'len(vars.token) == 0' # Run step only if var.token is not set\n    req:\n      /login:\n        post:\n          body:\n[...]\n```\n\n### `steps[*].loop:` `steps.\u003ckey\u003e.loop:`\n\nLoop setting for step.\n\n#### Simple loop step\n\n``` yaml\nsteps:\n  multicartin:\n    loop: 10\n    req:\n      /cart/in:\n        post:\n          body:\n            application/json:\n              product_id: \"{{ i }}\" # The loop count (0..9) is assigned to `i`.\n[...]\n```\n\nor\n\n``` yaml\nsteps:\n  multicartin:\n    loop:\n      count: 10\n    req:\n      /cart/in:\n        post:\n          body:\n            application/json:\n              product_id: \"{{ i }}\" # The loop count (0..9) is assigned to `i`.\n[...]\n```\n\n#### Retry step\n\nIt can be used as a retry mechanism by setting a condition in the `until:` section.\n\nIf the condition of `until:` is met, the loop is broken without waiting for the number of `count:` to be run.\n\nAlso, if the run of the number of `count:` completes but does not satisfy the condition of `until:`, then the step is considered to be failed.\n\n``` yaml\nsteps:\n  waitingroom:\n    loop:\n      count: 10\n      until: 'steps.waitingroom.res.status == \"201\"' # Store values of latest loop\n      minInterval: 500ms\n      maxInterval: 10 # sec\n      # jitter: 0.0\n      # interval: 5\n      # multiplier: 1.5\n    req:\n      /cart/in:\n        post:\n          body:\n[...]\n```\n\n### `steps[*].defer:` `steps.\u003ckey\u003e.defer:` [THIS IS EXPERIMENT]\n\nDeferring setting for step.\n\n```yaml\nsteps:\n  -\n    defer: true\n    req:\n      /cart:\n        delete:\n          body: null\n[...]\n```\n\nThe step marked `defer` behaves as follows.\n\n- If `defer: true` is set, run of the step is deferred until finish of the runbook.\n- Steps marked with `defer` are always run even if the running of intermediate steps fails.\n- If there are multiple steps marked with `defer`, they are run in LIFO order.\n    - Also, the included steps are added to run sequence of the parent runbook's deferred steps.\n\n### `steps[*].force:` `steps.\u003ckey\u003e.force:`\n\nForce step to run.\n\n```yaml\nsteps:\n[...]\n  -\n     force: true\n     dump: previous.res.body\n[...]\n```\n\n## Variables to be stored\n\nrunn can use variables and functions when running step.\n\nAlso, after step runs, HTTP responses, DB query results, etc. are automatically stored in variables.\n\nThe values are stored in predefined variables.\n\n| Variable name | Description |\n| --- | --- |\n| `vars` | Values set in the `vars:` section |\n| `steps` | Return values for each step |\n| `i` | Loop index (only in `loop:` section) |\n| `env` | Environment variables |\n| `current` | Return values of current step |\n| `previous` | Return values of previous step |\n| `parent` | Variables of parent runbook (only included) |\n\n## Runner\n\n### HTTP Runner: Do HTTP request\n\nUse `https://` or `http://` scheme to specify HTTP Runner.\n\nWhen the step is invoked, it sends the specified HTTP Request and records the response.\n\n``` yaml\nrunners:\n  req: https://example.com\nsteps:\n  -\n    desc: Post /users                     # description of step\n    req:                                  # key to identify the runner. In this case, it is HTTP Runner.\n      /users:                             # path of http request\n        post:                             # method of http request\n          headers:                        # headers of http request\n            Authorization: 'Bearer xxxxx'\n          body:                           # body of http request\n            application/json:             # Content-Type specification. In this case, it is \"Content-Type: application/json\"\n              username: alice\n              password: passw0rd\n          trace: false                    # add `X-Runn-Trace` header to HTTP request for tracing\n    test: |                               # test for current step\n      current.res.status == 201\n```\n\nSee [testdata/book/http.yml](testdata/book/http.yml) and [testdata/book/http_multipart.yml](testdata/book/http_multipart.yml).\n\n#### Structure of recorded responses\n\nThe following response\n\n```\nHTTP/1.1 200 OK\nContent-Length: 29\nContent-Type: application/json\nDate: Wed, 07 Sep 2022 06:28:20 GMT\nSet-Cookie: cookie-name=cookie-value\n\n{\"data\":{\"username\":\"alice\"}}\n```\n\nis recorded with the following structure.\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  res:\n    status: 200                              # current.res.status\n    headers:\n      Content-Length:\n        - '29'                               # current.res.headers[\"Content-Length\"][0]\n      Content-Type:\n        - 'application/json'                 # current.res.headers[\"Content-Type\"][0]\n      Date:\n        - 'Wed, 07 Sep 2022 06:28:20 GMT'    # current.res.headers[\"Date\"][0]\n      Set-Cookie:\n        - 'cookie-name=cookie-value'         # current.res.headers[\"Set-Cookie\"][0]\n    cookies:\n      cookie-name: *http.Cookie              # current.res.cookies[\"cookie-name\"].Value\n    body:\n      data:\n        username: 'alice'                    # current.res.body.data.username\n    rawBody: '{\"data\":{\"username\":\"alice\"}}' # current.res.rawBody\n```\n\n#### Do not follow redirect\n\nThe HTTP Runner interprets HTTP responses and automatically redirects.\nTo disable this, set `notFollowRedirect` to true.\n\n``` yaml\nrunners:\n  req:\n    endpoint: https://example.com\n    notFollowRedirect: true\n```\n\n#### Enable Cookie Sending\n\nThe HTTP Runner automatically saves cookies by interpreting HTTP responses.\nTo enable cookie sending during requests, set `useCookie` to true.\n\n``` yaml\nrunners:\n  req:\n    endpoint: https://example.com\n    useCookie: true\n```\n\nSee [testdata/book/cookie.yml](testdata/book/cookie.yml) and [testdata/book/cookie_in_requests_automatically.yml](testdata/book/cookie_in_requests_automatically.yml).\n\n#### Validation of HTTP request and HTTP response\n\nHTTP requests sent by `runn` and their HTTP responses can be validated.\n\n**OpenAPI v3:**\n\n``` yaml\nrunners:\n  myapi:\n    endpoint: https://api.example.com\n    openapi3: path/to/openapi.yaml\n    # skipValidateRequest: false\n    # skipValidateResponse: false\n    # skipCircularReferenceCheck: false # skip checking circular references in OpenAPIv3 document.\n```\n\n#### Custom CA and Certificates\n\n``` yaml\nrunners:\n  myapi:\n    endpoint: https://api.github.com\n    cacert: path/to/cacert.pem\n    cert: path/to/cert.pem\n    key: path/to/key.pem\n    # skipVerify: false\n```\n\n#### Add `X-Runn-Trace` header to HTTP request for tracing\n\n``` yaml\nrunners:\n  myapi:\n    endpoint: https://api.github.com\n    trace: true\n```\n\n### gRPC Runner: Do gRPC request\n\nUse `grpc://` scheme to specify gRPC Runner.\n\nWhen the step is invoked, it sends the specified gRPC Request and records the response.\n\n``` yaml\nrunners:\n  greq: grpc://grpc.example.com:80\nsteps:\n  -\n    desc: Request using Unary RPC                     # description of step\n    greq:                                             # key to identify the runner. In this case, it is gRPC Runner.\n      grpctest.GrpcTestService/Hello:                 # package.Service/Method of rpc\n        headers:                                      # headers of rpc\n          authentication: tokenhello\n        message:                                      # message of rpc\n          name: alice\n          num: 3\n          request_time: 2022-06-25T05:24:43.861872Z\n        trace: false                                  # add `x-runn-trace` header to gRPC request for tracing\n  -\n    desc: Request using Server streaming RPC\n    greq:\n      grpctest.GrpcTestService/ListHello:\n        headers:\n          authentication: tokenlisthello\n        message:\n          name: bob\n          num: 4\n          request_time: 2022-06-25T05:24:43.861872Z\n        timeout: 3sec                                 # timeout for rpc\n    test: |\n      steps.server_streaming.res.status == 0 \u0026\u0026 len(steps.server_streaming.res.messages) \u003e 0\n  -\n    desc: Request using Client streaming RPC\n    greq:\n      grpctest.GrpcTestService/MultiHello:\n        headers:\n          authentication: tokenmultihello\n        messages:                                     # messages of rpc\n          -\n            name: alice\n            num: 5\n            request_time: 2022-06-25T05:24:43.861872Z\n          -\n            name: bob\n            num: 6\n            request_time: 2022-06-25T05:24:43.861872Z\n```\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    tls: true\n    cacert: path/to/cacert.pem\n    cert: path/to/cert.pem\n    key: path/to/key.pem\n    # skipVerify: false\n    # importPaths:\n    #   - protobuf/proto\n    # protos:\n    #   - general/health.proto\n    #   - myapp/**/*.proto\n```\n\nSee [testdata/book/grpc.yml](testdata/book/grpc.yml).\n\n#### Structure of recorded responses\n\nThe following response\n\n```protocol-buffer\nmessage HelloResponse {\n  string message = 1;\n\n  int32 num = 2;\n\n  google.protobuf.Timestamp create_time = 3;\n}\n```\n\n```json\n{\"create_time\":\"2022-06-25T05:24:43.861872Z\",\"message\":\"hello\",\"num\":32}\n```\n\n\nand headers\n\n```yaml\ncontent-type: [\"application/grpc\"]\nhello: [\"this is header\"]\n```\n\nand trailers\n\n```yaml\nhello: [\"this is trailer\"]\n```\n\nare recorded with the following structure.\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  res:\n    status: 0                                      # current.res.status\n    headers:\n      content-type:\n        - 'application/grpc'                       # current.res.headers[0].content-type\n      hello:\n        - 'this is header'                         # current.res.headers[0].hello\n    trailers:\n      hello:\n        - 'this is trailer'                        # current.res.trailers[0].hello\n    message:\n      create_time: '2022-06-25T05:24:43.861872Z'   # current.res.message.create_time\n      message: 'hello'                             # current.res.message.message\n      num: 32                                      # current.res.message.num\n    messages:\n      -\n        create_time: '2022-06-25T05:24:43.861872Z' # current.res.messages[0].create_time\n        message: 'hello'                           # current.res.messages[0].message\n        num: 32                                    # current.res.messages[0].num\n```\n\n#### Add `x-runn-trace` header to gRPC request for tracing\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    trace: true\n```\n\n#### Buf\n\ngRPC Runner supports Buf ecosystem includes [Buf Schema Registry](https://buf.build/product/bsr).\n\nIt can use the buf modules ( and protos ) it depends on.\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    bufDirs:\n      - path/to # Set buf directories for registering buf modules and protos\n```\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    bufLocks:\n      - path/to/buf.lock # Register buf modules using buf.lock\n```\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    bufConfigs:\n      - path/to/buf.yaml # Register buf modules using buf.yaml\n```\n\n``` yaml\nrunners:\n  greq:\n    addr: grpc.example.com:8080\n    bufModules:\n        - buf.build/owner/repository\n        - buf.build/owner2/repository2\n```\n\n### DB Runner: Query a database\n\nUse dsn (Data Source Name) to specify DB Runner.\n\nWhen step is invoked, it executes the specified query the database.\n\n``` yaml\nrunners:\n  db: postgres://dbuser:dbpass@hostname:5432/dbname\nsteps:\n  -\n    desc: Select users            # description of step\n    db:                           # key to identify the runner. In this case, it is DB Runner.\n      query: SELECT * FROM users; # query to execute\n      trace: false                # add comment with trace token to query for tracing\n```\n\nSee [testdata/book/db.yml](testdata/book/db.yml).\n\n#### Structure of recorded responses\n\nIf the query is a SELECT clause, it records the selected `rows`,\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  rows:\n    -\n      id: 1                           # current.rows[0].id\n      username: 'alice'               # current.rows[0].username\n      password: 'passw0rd'            # current.rows[0].password\n      email: 'alice@example.com'      # current.rows[0].email\n      created: '2017-12-05T00:00:00Z' # current.rows[0].created\n    -\n      id: 2                           # current.rows[1].id\n      username: 'bob'                 # current.rows[1].username\n      password: 'passw0rd'            # current.rows[1].password\n      email: 'bob@example.com'        # current.rows[1].email\n      created: '2022-02-22T00:00:00Z' # current.rows[1].created\n```\n\notherwise it records `last_insert_id` and `rows_affected` .\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  last_insert_id: 3 # current.last_insert_id\n  rows_affected: 1  # current.rows_affected\n```\n\n#### Add comment with trace token to query for tracing\n\n``` yaml\nrunners:\n  db:\n    dsn: mysql://dbuser:dbpass@hostname:3306/dbname\n    trace: true\n```\n\n#### Support Databases\n\n**PostgreSQL:**\n\n``` yaml\nrunners:\n  mydb: postgres://dbuser:dbpass@hostname:5432/dbname\n```\n\n``` yaml\nrunners:\n  db: pg://dbuser:dbpass@hostname:5432/dbname\n```\n\n**MySQL:**\n\n``` yaml\nrunners:\n  testdb: mysql://dbuser:dbpass@hostname:3306/dbname\n```\n\n``` yaml\nrunners:\n  db: my://dbuser:dbpass@hostname:3306/dbname\n```\n\n**SQLite3:**\n\n``` yaml\nrunners:\n  db: sqlite:///path/to/dbname.db\n```\n\n``` yaml\nrunners:\n  local: sq://dbname.db\n```\n\n**Cloud Spanner:**\n\n``` yaml\nrunners:\n  testdb: spanner://test-project/test-instance/test-database\n```\n\n``` yaml\nrunners:\n  db: sp://test-project/test-instance/test-database\n```\n\n### CDP Runner: Control browser using Chrome DevTools Protocol (CDP)\n\nUse `cdp://` or `chrome://` scheme to specify CDP Runner.\n\nWhen the step is invoked, it controls browser via Chrome DevTools Protocol.\n\n``` yaml\nrunners:\n  cc: chrome://new\nsteps:\n  -\n    desc: Navigate, click and get h1 using CDP  # description of step\n    cc:                                         # key to identify the runner. In this case, it is CDP Runner.\n      actions:                                  # actions to control browser\n        - navigate: https://pkg.go.dev/time\n        - click: 'body \u003e header \u003e div.go-Header-inner \u003e nav \u003e div \u003e ul \u003e li:nth-child(2) \u003e a'\n        - waitVisible: 'body \u003e footer'\n        - text: 'h1'\n  -\n    test: |\n      previous.text == 'Install the latest version of Go'\n```\n\nSee [testdata/book/cdp.yml](testdata/book/cdp.yml).\n\n#### Functions for action to control browser\n\n\u003c!-- repin:fndoc --\u003e\n**`attributes`** (aliases: `getAttributes`, `attrs`, `getAttrs`)\n\nGet the element attributes for the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - attributes:\n      sel: 'h1'\n# record to current.attrs:\n```\n\nor\n\n```yaml\nactions:\n  - attributes: 'h1'\n```\n\n**`click`**\n\nSend a mouse click event to the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - click:\n      sel: 'nav \u003e div \u003e a'\n```\n\nor\n\n```yaml\nactions:\n  - click: 'nav \u003e div \u003e a'\n```\n\n**`doubleClick`**\n\nSend a mouse double click event to the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - doubleClick:\n      sel: 'nav \u003e div \u003e li'\n```\n\nor\n\n```yaml\nactions:\n  - doubleClick: 'nav \u003e div \u003e li'\n```\n\n**`evaluate`** (aliases: `eval`)\n\nEvaluate the Javascript expression (`expr`).\n\n```yaml\nactions:\n  - evaluate:\n      expr: 'document.querySelector(\"h1\").textContent = \"hello\"'\n```\n\nor\n\n```yaml\nactions:\n  - evaluate: 'document.querySelector(\"h1\").textContent = \"hello\"'\n```\n\n**`fullHTML`** (aliases: `getFullHTML`, `getHTML`, `html`)\n\nGet the full html of page.\n\n```yaml\nactions:\n  - fullHTML\n# record to current.html:\n```\n\n**`innerHTML`** (aliases: `getInnerHTML`)\n\nGet the inner html of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - innerHTML:\n      sel: 'h1'\n# record to current.html:\n```\n\nor\n\n```yaml\nactions:\n  - innerHTML: 'h1'\n```\n\n**`latestTab`** (aliases: `latestTarget`)\n\nChange current frame to latest tab.\n\n```yaml\nactions:\n  - latestTab\n```\n\n**`localStorage`** (aliases: `getLocalStorage`)\n\nGet localStorage items.\n\n```yaml\nactions:\n  - localStorage:\n      origin: 'https://github.com'\n# record to current.items:\n```\n\nor\n\n```yaml\nactions:\n  - localStorage: 'https://github.com'\n```\n\n**`location`** (aliases: `getLocation`)\n\nGet the document location.\n\n```yaml\nactions:\n  - location\n# record to current.url:\n```\n\n**`navigate`**\n\nNavigate the current frame to `url` page.\n\n```yaml\nactions:\n  - navigate:\n      url: 'https://pkg.go.dev/time'\n```\n\nor\n\n```yaml\nactions:\n  - navigate: 'https://pkg.go.dev/time'\n```\n\n**`outerHTML`** (aliases: `getOuterHTML`)\n\nGet the outer html of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - outerHTML:\n      sel: 'h1'\n# record to current.html:\n```\n\nor\n\n```yaml\nactions:\n  - outerHTML: 'h1'\n```\n\n**`screenshot`** (aliases: `getScreenshot`)\n\nTake a full screenshot of the entire browser viewport.\n\n```yaml\nactions:\n  - screenshot\n# record to current.png:\n```\n\n**`scroll`** (aliases: `scrollIntoView`)\n\nScroll the window to the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - scroll:\n      sel: 'body \u003e footer'\n```\n\nor\n\n```yaml\nactions:\n  - scroll: 'body \u003e footer'\n```\n\n**`sendKeys`**\n\nSend keys (`value`) to the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - sendKeys:\n      sel: 'input[name=username]'\n      value: 'k1lowxb@gmail.com'\n```\n\n**`sessionStorage`** (aliases: `getSessionStorage`)\n\nGet sessionStorage items.\n\n```yaml\nactions:\n  - sessionStorage:\n      origin: 'https://github.com'\n# record to current.items:\n```\n\nor\n\n```yaml\nactions:\n  - sessionStorage: 'https://github.com'\n```\n\n**`setUploadFile`** (aliases: `setUpload`)\n\nSet upload file (`path`) to the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - setUploadFile:\n      sel: 'input[name=avator]'\n      path: '/path/to/image.png'\n```\n\n**`setUserAgent`** (aliases: `setUA`, `ua`, `userAgent`)\n\nSet the default User-Agent\n\n```yaml\nactions:\n  - setUserAgent:\n      userAgent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36'\n```\n\nor\n\n```yaml\nactions:\n  - setUserAgent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36'\n```\n\n**`submit`**\n\nSubmit the parent form of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - submit:\n      sel: 'form.login'\n```\n\nor\n\n```yaml\nactions:\n  - submit: 'form.login'\n```\n\n**`text`** (aliases: `getText`)\n\nGet the visible text of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - text:\n      sel: 'h1'\n# record to current.text:\n```\n\nor\n\n```yaml\nactions:\n  - text: 'h1'\n```\n\n**`textContent`** (aliases: `getTextContent`)\n\nGet the text content of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - textContent:\n      sel: 'h1'\n# record to current.text:\n```\n\nor\n\n```yaml\nactions:\n  - textContent: 'h1'\n```\n\n**`title`** (aliases: `getTitle`)\n\nGet the document `title`.\n\n```yaml\nactions:\n  - title\n# record to current.title:\n```\n\n**`value`** (aliases: `getValue`)\n\nGet the Javascript value field of the first element node matching the selector (`sel`).\n\n```yaml\nactions:\n  - value:\n      sel: 'input[name=address]'\n# record to current.value:\n```\n\nor\n\n```yaml\nactions:\n  - value: 'input[name=address]'\n```\n\n**`wait`** (aliases: `sleep`)\n\nWait for the specified `time`.\n\n```yaml\nactions:\n  - wait:\n      time: '10sec'\n```\n\nor\n\n```yaml\nactions:\n  - wait: '10sec'\n```\n\n**`waitReady`**\n\nWait until the element matching the selector (`sel`) is ready.\n\n```yaml\nactions:\n  - waitReady:\n      sel: 'body \u003e footer'\n```\n\nor\n\n```yaml\nactions:\n  - waitReady: 'body \u003e footer'\n```\n\n**`waitVisible`**\n\nWait until the element matching the selector (`sel`) is visible.\n\n```yaml\nactions:\n  - waitVisible:\n      sel: 'body \u003e footer'\n```\n\nor\n\n```yaml\nactions:\n  - waitVisible: 'body \u003e footer'\n```\n\n\n\u003c!-- repin:fndoc --\u003e\n\n### SSH Runner: execute commands on a remote server connected via SSH\n\nUse `ssh://` scheme to specify SSH Runner.\n\nWhen step is invoked, it executes commands on a remote server connected via SSH.\n\n``` yaml\nrunners:\n  sc: ssh://username@hostname:port\nsteps:\n  -\n    desc: 'execute `hostname`' # description of step\n    sc:\n      command: hostname\n```\n\n\n``` yaml\nrunners:\n  sc:\n    hostname: hostname\n    user: username\n    port: 22\n    # host: myserver\n    # sshConfig: path/to/ssh_config\n    # keepSession: false\n    # localForward: '33306:127.0.0.1:3306'\n    # keyboardInteractive:\n    #   - match: Username\n    #     answer: k1low\n    #   - match: OTP\n    #     answer: ${MY_OTP}\n```\n\nSee [testdata/book/sshd.yml](testdata/book/sshd.yml).\n\n#### Structure of recorded responses\n\nThe response to the run command is always `stdout` and `stderr`.\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  stdout: 'hello world' # current.stdout\n  stderr: ''            # current.stderr\n```\n\n### Exec Runner: execute command\n\n\u003e **Note**\n\u003e Exec runner requires `run:exec` scope to run.\n\nThe `exec` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nIt execute command using `command:`, `stdin:`, `shell:`, `background:` and `liveOutput:`.\n\n``` yaml\n-\n  exec:\n    command: grep hello\n    stdin: '{{ steps[3].res.rawBody }}'\n```\n\n``` yaml\n-\n  exec:\n    command: echo $0\n    shell: bash\n```\n\n`background:` set to `true` to run the command in the background.\n\n``` yaml\n-\n  exec:\n    command: kubectl port-forward svc/nginx 8080:80\n    liveOutput: true\n```\n\n`liveOutput:` set to `true` to output the command output live.\n\n``` yaml\n-\n  exec:\n    command: |\n      echo \"Start heavy commands\"\n      sleep 5\n      echo \"Heavy command finished\"\n    liveOutput: true\n```\n\nSee [testdata/book/exec.yml](testdata/book/exec.yml).\n\n#### Structure of recorded responses\n\nThe response to the run command is always `stdout`, `stderr` and `exit_code`.\n\n``` yaml\n[`step key` or `current` or `previous`]:\n  stdout: 'hello world' # current.stdout\n  stderr: ''            # current.stderr\n  exit_code: 0          # current.exit_code\n```\n\n#### `exec.shell:`\n\nUse `shell:` to define the shell and options to be used by the Exec runner.\n\n| Parameter| Command run internally |\n|:----|:----|\n| unspecified | `bash -e -c {0}` |\n| `bash` | `bash --noprofile --norc -eo pipefail -c {0}` |\n| `sh` | `sh -e -c {0}` |\n\n### Test Runner: test using recorded values\n\nThe `test` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nIt evaluates the conditional expression using the recorded values.\n\n``` yaml\n-\n  test: steps[3].res.status == 200\n```\n\nThe `test` runner can run in the same steps as the other runners.\n\n### Dump Runner: dump recorded values\n\nThe `dump` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nIt dumps the specified recorded values.\n\n``` yaml\n-\n  dump: steps[4].rows\n```\n\nor\n\n``` yaml\n-\n  dump:\n    expr: steps[4].rows\n    out: path/to/dump.out\n    disableTrailingNewline: true # disable trailing newline. default is false\n    disableMaskingSecrets: true  # disable masking secrets. default is false\n```\n\nThe `dump` runner can run in the same steps as the other runners.\n\n### Include Runner: include other runbook\n\nThe `include` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nInclude runner reads and runs the runbook in the specified path.\n\nRecorded values are nested.\n\n``` yaml\n-\n  include: path/to/get_token.yml\n```\n\nIt is also possible to override `vars:` of included runbook.\n\n``` yaml\n-\n  include:\n    path: path/to/login.yml\n    vars:\n      username: alice\n      password: alicepass\n-\n  include:\n    path: path/to/login.yml\n    vars:\n      username: bob\n      password: bobpass\n```\n\nIt is also possible to skip all `test:` sections in the included runbook.\n\n``` yaml\n-\n  include:\n    path: path/to/signup.yml\n    skipTest: true\n```\n\nIt is also possible to force all steps in the included runbook to run.\n\n``` yaml\n-\n  include:\n    path: path/to/signup.yml\n    force: true\n```\n\n### Bind Runner: bind variables\n\nThe `bind` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nIt bind runner binds any values with another key.\n\n``` yaml\n  -\n    req:\n      /users/k1low:\n        get:\n          body: null\n  -\n    bind:\n      user_id: steps[0].res.body.data.id\n  -\n    dump: user_id\n```\n\nThe `bind` runner can run in the same steps as the other runners.\n\n### Runner Runner: Define runner in the middle of steps.\n\nThe `runner` runner is a built-in runner, so there is no need to specify it in the `runners:` section.\n\nIt defines a runner in the middle of steps.\n\n``` yaml\n  -\n    runner:\n      sc: ssh://username@hostname:port\n  -\n    sc:\n      command: hostname\n```\n\nThe `runner` runner can not run in the same steps as the other runners.\n\n## Expression evaluation engine\n\nrunn has embedded [expr-lang/expr](https://github.com/expr-lang/expr) as the evaluation engine for the expression.\n\nSee [Language Definition](https://expr-lang.org/docs/language-definition).\n\n### Additional built-in functions\n\n- `urlencode` ... [url.QueryEscape](https://pkg.go.dev/net/url#QueryEscape)\n- `bool` ... [cast.ToBool](https://pkg.go.dev/github.com/spf13/cast#ToBool)\n- `compare` ... Compare two values ( `func(x, y any, ignorePaths ...string) bool` ). The optional `ignorePaths` argument is a list of [jq syntax path expressions](https://jqlang.github.io/jq/manual/#path) to ignore when comparing two values.\n- `diff` ... Difference between two values ( `func(x, y any, ignorePaths ...string) string` ). The optional `ignorePaths` argument is a list of [jq syntax path expressions](https://jqlang.github.io/jq/manual/#path) to ignore when comparing two values.\n- `pick` ... Returns same map type filtered by given keys left [lo.PickByKeys](https://github.com/samber/lo?tab=readme-ov-file#pickbykeys).\n- `omit` ... Returns same map type filtered by given keys excluded [lo.OmitByKeys](https://github.com/samber/lo?tab=readme-ov-file#omitbykeys).\n- `merge` ... Merges multiple maps from left to right [lo.Assign](https://github.com/samber/lo?tab=readme-ov-file#assign).\n- `input` ... [prompter.Prompt](https://pkg.go.dev/github.com/Songmu/prompter#Prompt)\n- `intersect` ... Find the intersection of two iterable values ( `func(x, y any) any` ).\n- `secret` ... [prompter.Password](https://pkg.go.dev/github.com/Songmu/prompter#Password)\n- `select` ... Select from candidates. `func(message string, candidates []string, default string) string`\n- `basename` ... [filepath.Base](https://pkg.go.dev/path/filepath#Base)\n- `time` ... Converts the given string or number to `time.Time{}`.\n- `faker.*` ... Generate fake data using [Faker](https://pkg.go.dev/github.com/k1LoW/runn/internal/builtin#Faker) ).\n\n## Option\n\nSee https://pkg.go.dev/github.com/k1LoW/runn#Option\n\n### Example: Run as a test helper ( func `T` )\n\nhttps://pkg.go.dev/github.com/k1LoW/runn#T\n\n``` go\no, err := runn.Load(\"testdata/**/*.yml\", runn.T(t))\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.RunN(ctx); err != nil {\n\tt.Fatal(err)\n}\n```\n\n### Example: Add custom function ( func `Func` )\n\nhttps://pkg.go.dev/github.com/k1LoW/runn#Func\n\n``` yaml\ndesc: Test using GitHub\nrunners:\n  req:\n    endpoint: https://github.com\nsteps:\n  -\n    req:\n      /search?l={{ urlencode('C++') }}\u0026q=runn\u0026type=Repositories:\n        get:\n          body:\n            application/json:\n              null\n    test: 'steps[0].res.status == 200'\n```\n\n``` go\no, err := runn.Load(\"testdata/**/*.yml\", runn.Func(\"urlencode\", url.QueryEscape))\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.RunN(ctx); err != nil {\n\tt.Fatal(err)\n}\n```\n\n## Scope\n\nrunn requires explicit specification of scope for some features.\n\nrunn has the following scopes.\n\n| Scope | Description | Default |\n| --- | --- | --- |\n| `read:parent` | Required for reading files above the working directory. | `false` |\n| `read:remote` | Required for reading remote files. | `false` |\n| `run:exec` | Required for running Exec runner. | `false` |\n\nTo specify scopes, using the `--scopes` option or the environment variable `RUNN_SCOPES`.\n\n```console\n$ runn run path/to/**/*.yml --scopes read:parent,read:remote\n```\n\n```console\n$ env RUNN_SCOPES=read:parent,read:remote runn run path/to/**/*.yml\n```\n\nAlso, [runn.Scopes](https://pkg.go.dev/github.com/k1LoW/runn#Scopes) can be used in the code\n\n``` go\no, err := runn.Load(\"path/to/**/*.yml\", runn.Scopes(runn.ScopeAllowReadParent, runn.ScopeAllowReadRemote))\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.RunN(ctx); err != nil {\n\tt.Fatal(err)\n}\n```\n\nTo disable scope, can use `!read:*` instead of `read:*`\n\n```console\n$ runn run path/to/**/*.yml --scopes '!read:parent'\n```\n\n## Filter runbooks to be executed by the environment variable `RUNN_RUN`\n\nRun only runbooks matching the filename \"login\".\n\n``` console\n$ env RUNN_RUN=login go test ./... -run TestRouter\n```\n\n## Measure elapsed time as profile\n\n``` go\nopts := []runn.Option{\n\trunn.T(t),\n\trunn.Book(\"testdata/books/login.yml\"),\n\trunn.Profile(true),\n}\no, err := runn.New(opts...)\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.Run(ctx); err != nil {\n\tt.Fatal(err)\n}\nf, err := os.Open(\"profile.json\")\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.DumpProfile(f); err != nil {\n\tt.Fatal(err)\n}\n```\n\nor\n\n``` console\n$ runn run testdata/books/login.yml --profile\n```\n\nThe runbook run profile can be read with `runn rprof` command.\n\n``` console\n$ runn rprof runn.prof\n  runbook[login site](t/b/login.yml)           2995.72ms\n    steps[0].req                                747.67ms\n    steps[1].req                                185.69ms\n    steps[2].req                                192.65ms\n    steps[3].req                                188.23ms\n    steps[4].req                                569.53ms\n    steps[5].req                                299.88ms\n    steps[6].test                                 0.14ms\n    steps[7].include                            620.88ms\n      runbook[include](t/b/login_include.yml)   605.56ms\n        steps[0].req                            605.54ms\n    steps[8].req                                190.92ms\n  [total]                                      2995.84ms\n```\n\n## Capture runbook runs\n\n``` go\nopts := []runn.Option{\n\trunn.T(t),\n\trunn.Capture(capture.Runbook(\"path/to/dir\")),\n}\no, err := runn.Load(\"testdata/books/**/*.yml\", opts...)\nif err != nil {\n\tt.Fatal(err)\n}\nif err := o.RunN(ctx); err != nil {\n\tt.Fatal(err)\n}\n```\n\nor\n\n``` console\n$ runn run path/to/**/*.yml --capture path/to/dir\n```\n\n## Load test using runbooks\n\nYou can use the `runn loadt` command for load testing using runbooks.\n\n``` console\n$ runn loadt --load-concurrent 2 --max-rps 0 path/to/*.yml\n\nNumber of runbooks per RunN....: 15\nWarm up time (--warm-up).......: 5s\nDuration (--duration)..........: 10s\nConcurrent (--load-concurrent).: 2\nMax RunN per second (--max-rps): 0\n\nTotal..........................: 12\nSucceeded......................: 12\nFailed.........................: 0\nError rate.....................: 0%\nRunN per seconds...............: 1.2\nLatency .......................: max=1,835.1ms min=1,451.3ms avg=1,627.8ms med=1,619.8ms p(90)=1,741.5ms p(99)=1,788.4ms\n\n```\n\nIt also checks the results of the load test with the `--threshold` option. If the condition is not met, it returns exit status 1.\n\n``` console\n$ runn loadt --load-concurrent 2 --max-rps 0 --threshold 'error_rate \u003c 10' path/to/*.yml\n\nNumber of runbooks per RunN...: 15\nWarm up time (--warm-up)......: 5s\nDuration (--duration).........: 10s\nConcurrent (--load-concurrent): 2\n\nTotal.........................: 13\nSucceeded.....................: 12\nFailed........................: 1\nError rate....................: 7.6%\nRunN per seconds..............: 1.3\nLatency ......................: max=1,790.2ms min=95.0ms avg=1,541.4ms med=1,640.4ms p(90)=1,749.7ms p(99)=1,786.5ms\n\nError: (error_rate \u003c 10) is not true\nerror_rate \u003c 10\n├── error_rate =\u003e 14.285714285714285\n└── 10 =\u003e 10\n```\n\n### Variables for threshold\n\n| Variable name | Type | Description |\n| --- | --- | --- |\n| `total` | `int` | Total |\n| `succeeded` | `int` | Succeeded |\n| `failed` | `int` | Failed |\n| `error_rate` | `float` | Error rate |\n| `rps` | `float` | RunN per seconds |\n| `max` | `float` | Latency max (ms) |\n| `mid` | `float` | Latency mid (ms) |\n| `min` | `float` | Latency min (ms) |\n| `p90` | `float` | Latency p(90) (ms) |\n| `p99` | `float` | Latency p(99) (ms) |\n| `avg` | `float` | Latency avg (ms) |\n\n## Install\n\n### As a CLI tool\n\n**deb:**\n\n``` console\n$ export RUNN_VERSION=X.X.X\n$ curl -o runn.deb -L https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.deb\n$ dpkg -i runn.deb\n```\n\n**RPM:**\n\n``` console\n$ export RUNN_VERSION=X.X.X\n$ yum install https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.rpm\n```\n\n**apk:**\n\n``` console\n$ export RUNN_VERSION=X.X.X\n$ curl -o runn.apk -L https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.apk\n$ apk add runn.apk\n```\n\n**homebrew tap:**\n\n```console\n$ brew install k1LoW/tap/runn\n```\n\n**[aqua](https://aquaproj.github.io/):**\n\n```console\n$ aqua g -i k1LoW/runn\n```\n\n**manually:**\n\nDownload binary from [releases page](https://github.com/k1LoW/runn/releases)\n\n**docker:**\n\n```console\n$ docker container run -it --rm --name runn -v $PWD:/books ghcr.io/k1low/runn:latest list /books/*.yml\n```\n\n**go install:**\n\n```console\n$ go install github.com/k1LoW/runn/cmd/runn@latest\n```\n\n### As a test helper\n\n```console\n$ go get github.com/k1LoW/runn\n```\n\n## Alternatives\n\n- [zoncoen/scenarigo](https://github.com/zoncoen/scenarigo): An end-to-end scenario testing tool for HTTP/gRPC server.\n\n## References\n\n- [zoncoen/scenarigo](https://github.com/zoncoen/scenarigo): An end-to-end scenario testing tool for HTTP/gRPC server.\n- [fullstorydev/grpcurl](https://github.com/fullstorydev/grpcurl): Like cURL, but for gRPC: Command-line tool for interacting with gRPC servers\n- [ktr0731/evans](https://github.com/ktr0731/evans): Evans: more expressive universal gRPC client\n\n## License\n\n- [MIT License](LICENSE)\n    - Include logo as well as source code.\n    - Only logo license can be selected [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).\n    - Also, if there is no alteration to the logo and it is used for technical information about runn, I would not say anything if the copyright notice is omitted.\n","funding_links":["https://github.com/sponsors/k1LoW"],"categories":["Automated API Testing \u0026 Load Testing Tools","Automated Testing"],"sub_categories":["46. [runn](https://github.com/k1LoW/runn)"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk1LoW%2Frunn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fk1LoW%2Frunn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk1LoW%2Frunn/lists"}