{"id":16800484,"url":"https://github.com/undera/encarno","last_synced_at":"2025-10-11T19:32:11.965Z","repository":{"id":37095290,"uuid":"497043950","full_name":"undera/encarno","owner":"undera","description":"Load generator for HTTP with high throughput and high precision","archived":false,"fork":false,"pushed_at":"2023-10-03T14:44:23.000Z","size":207,"stargazers_count":28,"open_issues_count":0,"forks_count":3,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-18T06:51:36.575Z","etag":null,"topics":["http","load-generator","load-testing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/undera.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-27T15:17:34.000Z","updated_at":"2024-04-04T12:25:15.000Z","dependencies_parsed_at":"2023-10-03T21:53:27.876Z","dependency_job_id":"9b19d726-3463-405b-9d68-cbf2c4218d71","html_url":"https://github.com/undera/encarno","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/undera%2Fencarno","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/undera%2Fencarno/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/undera%2Fencarno/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/undera%2Fencarno/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/undera","download_url":"https://codeload.github.com/undera/encarno/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244897834,"owners_count":20528297,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["http","load-generator","load-testing"],"created_at":"2024-10-13T09:33:29.511Z","updated_at":"2025-10-11T19:32:06.924Z","avatar_url":"https://github.com/undera.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Encarno - The Efficient Load Generator\n\nThe name comes from portuguese _[encarno](https://en.wiktionary.org/wiki/encarno) (/əŋˈkar.nu/)_ and means\nroughly \"[I impersonate](#history)\".\n\n## Key Features\n\n- HTTP 1.1 protocol testing, TLS supported, IPv6 supported\n- flexible load profiles in [\"open\" and \"closed\" workload](https://www.google.com/search?q=open+closed+workload) modes\n- accurate load generating up to tens of thousands hits/s\n- precise result measurements of nanosecond resolution\n- efficient and low overhead (written in Go)\n- minimalistic scripting, with regex extracts and asserts\n\n\n## Usage as Taurus Module\n\nThe easiest way to get started is to install [the Python package](https://pypi.org/project/encarno/) using `pip`, which will install also Taurus if needed:\n\n```shell\npip install encarno\n```\n\nAfter that, running any test with `executor: encarno` will automatically download the appropriate version of the Encarno binary. In case you need to point the tool to a custom binary, use this config snippet:\n```yaml\nmodules:\n  encarno: \n    path: /my/custom/encarno\n```\n\nTo run the test, use the usual Taurus command-line with config files. See below for the config examples:\n```shell\nbzt my-config-with-encarno.yml\n```\n\n[Docker image](https://hub.docker.com/r/undera/encarno) is also available for containerized environments: \n\n```shell\ndocker run -it -v `pwd`:/conf undera/encarno /conf/config.yml\n```\n\n### Closed Workload Mode\n\nClosed workload is the load testing mode when relatively small pool of workers hit the service _as fast as they can_. As service reaches the bottleneck, the response time grows and workers produce less and less hits per second. This kind of workload is typical for service-to-service communications inside cluster. \n\nIn typical tests, the number of workers gradually increases over time to reveal the _capacity limit_ of the service. The result of such test is a _scalability profile_ for the service, also offering the estimation of throughput limits for the [open workload](#open-workload-mode) tests. \n\nThe Taurus config file for closed workload using Encarno:\n\n```yaml\n---\nexecution:\n  - executor: encarno\n    scenario: simple\n    \n    concurrency: 50\n    ramp-up: 5m\n    # steps: 10  # breaks ramp-up into N flat steps\n\nscenarios:\n  simple:\n    requests: \n      - http://service.net:8080/api/path\n```\n\nNote that `hold-for` and `iterations` load profile options are also supported, if you need them. Scenario definition can be [as sophisticated as you need it](#scripting-capabilities).\n\n### Open Workload Mode\n\nOpen workload reflects public service scenario, when the number of clients is so big that slowing responses do not lead to decrease in service requests. This is achieved in tests by using large pool of workers that hit service according to _requests schedule_. Usually, that schedule is growing linearly, to reveal the breaking point of the service. Or a steady rate is applied to measure _performance quality characteristics_ for the service, such as response time percentiles.\n\nThe main value we configure for open workload tests is the `throughput`, which is the number of requests per second to perform. For the breaking point (aka _stress test_) scenarios we configure it above the [capacity limit](#closed-workload-mode) (~factor x1.5), for quality measurement we aim below the limit (~factor 1/2 or 80%). Usually we also put some limit on possible worker count `concurrency`, due to RAM/CPU being finite for load generator machine.\n\nStress test config example:\n\n```yaml\n---\nexecution:\n  - executor: encarno\n    scenario: simple\n    \n    concurrency: 5000  # it's now the limit, not desired level\n    \n    throughput: 25000  # hit/s beyond server's capacity\n    ramp-up: 5m\n    # steps: 10  # breaks ramp-up into N flat steps\n\nscenarios:\n  simple:\n    requests: \n      - http://service.net:8080/api/path\n```\n\nQuality measurement config:\n\n```yaml\n---\nexecution:\n  - executor: encarno\n    scenario: simple\n\n    concurrency: 5000  # it's now the limit, not desired level\n\n    throughput: 10000  # hit/s below breaking point\n    ramp-up: 1m        # should be much shorter than hold-for\n    hold-for: 20m      # enough time to accumulate statistics\n\nscenarios:\n  simple:\n    requests: \n      - http://service.net:8080/api/path\n```\n\n### Scripting Capabilities\n\nThere are 3 ways to specify inputs for Encarno test: YAML definition via `requests`, URLs list via `requests` and externally generated input file. \n\n#### Requests Defined in YAML \nThe YAML definition looks like [typical Taurus script](https://gettaurus.org/docs/ExecutionSettings/#Scenario) in `requests` section under `scenarios`:\n\n```yaml\nscenarios:\n  simple:\n    default-address: http://i-am-used-by-default:8000\n    timeout: 5s\n\n    variables:\n      var1: someval\n\n    headers:\n      X-My-Global-Header: for all requests\n\n    requests:\n      - /assumes-default-address\n      - https://full-url-possible/here\n\n      - label: extended detailed request\n        url: /path\n        method: POST\n        headers:\n          X-One-More-Header: or many\n          Content-Type: application/json\n        body: '{\"can be\": \"like this\"}'\n        # body-file: some.json  # alternative to inline `body`\n\n      - label: regex features, source\n        url: /?variable=${var1}\n        extract-regexp:\n          etag: 'ETag: (\".+\")'\n          date: 'Date: ([^\\n]+)'        \n\n      - label: regex features, destination\n        url: /path\n        method: POST\n        headers:\n          If-None-Match: '${etag}'\n        body: \"date was ${date}\"\n        assert:\n          - 'HTTP/1.1 200 OK'\n          - not: true\n            contains:\n              - 'error'\n```\n\nNote that `timeout` is only supported on the global level, affecting all the requests equally.\n\nThe `variables`, `assert` and `extract-regexp` features work on the full request and response payload text, without breakdown into URI/status/headers/body. Note that variable and regexp usage _will_ make your tests to work a bit slower, due to the processing overhead. Also some more RAM will be used by the load generator.\n\nFor HTTP, there is special `:content-length:` variable to be used to obtain correct body length when variables usage alters it dynamically. Taurus module will automatically use that variable when generating POST requests.\n\n#### URLs From Text File\n\nThere are the cases when you have a long list of URLs parsed from `access.log` of your server, or dumped from database, or generated by some script. In that situation, you can specify the file containing URLs as value for `requests` option:\n\n```yaml\nscenarios:\n  simple:\n    default-address: http://i-am-used-by-default:8000\n    timeout: 5s\n    \n    headers:\n      X-My-Global-Header: for all requests\n      \n    requests: urls-file.txt\n```\n\nThe format of the `urls-file.txt` is trivial. It can be either the list of URLs, one per line, or it can also contain request `label` per URL, divided by space:\n```text\n/just-url/assumes-default-address\nhttp://full-url/goes/here\nthe_label_before_space /and/then/url\n```\n\nAll the global settings from scenario still apply to URLs file case, the HTTP method is assumed to be `GET`. Variable evaluation also works for this kind of input.\n\n#### Custom Payload File\n\nFinally, if you need full control over what is sent over network, you can use script file in Encarno's internal [_payload input_ format](#payload-input-format), which ignores most of other scenario options:\n\n```yaml\nscenarios:\n  simple:\n    script: custom_input.enc\n    # input-strings: custom_input.str  # for the indexed strings, if needed\n```\n\n### TLS Configuration\n\nFor the cases, when connecting to server needs special TLS settings like custom cipher suites or TLS versions, please use the following config snippet:\n\n```yaml\nmodules:\n  encarno:\n    tls-config:\n      insecureskipverify: true  # allow self-signed and invalid certificates, default: false\n      minversion: 771  # min version of TLS, default is 1.2\n      maxversion: 772  # max version of TLS, default is 1.3\n      tlsciphersuites: # list of cipher suite names to use, optional\n        - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA\n        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384\n        - TLS_RSA_WITH_AES_128_CBC_SHA\n        - TLS_RSA_WITH_AES_256_GCM_SHA384\n```\n\nPossible values for TLS version:\n - TLS 1.3 = `772`\n - TLS 1.2 = `771`\n - TLS 1.1 = `770`\n - TLS 1.0 = `769`\n\n### Debug Trace Log\n\nIf you want to see what exactly were sent to the server and what was the response, you may enable the detailed trace log. Encarno would write `encarno_trace.txt` file, containing all meta information, request and response payloads:\n\n```yaml\nmodules:\n  encarno:\n    trace-level: 500  # defaults to 1000\n```\n\nThe trace option logic is \"log records which status code is greater or equal X\". All network level errors have special code `999`. Be careful with trace level, it can produce large files quickly and run you out of disk space. It is recommended to only enable tracing for certain kind of errors. \n\nSome level examples:\n\n- `999` or `600` - only the network level errors\n- `500` - all 5xx server errors plus network level errors\n- `400` - client-side, server-side, and network level\n- `0` - dump all the traffic\n- `1000` - don't write trace file\n\n\n### Sidebar Widget\n\nWhen Taurus displays [console dashboard](https://gettaurus.org/docs/ConsoleReporter/), Encarno provides additional information in its sidebar widget:\n- `X wait` - the number of workers that are waiting to get input payload/schedule, if above zero - your load generator is at capacity\n- `X busy` - the number of workers actively doing job, synonym to concurrency\n- `X sleep` - number of workers waiting for the right time to query (only for open workload), if zero - your load generator is likely at capacity\n- `X lag` - the average lag between scheduled time to request and actual time (only for open workload), if above zero - your load generator is likely at capacity \n\nPlease note that widget information may be ahead of aggregate statistics, due to Taurus reporting facility still crunching numbers.\n\n## Standalone Usage\n\nEncarno tool is designed to be used as part of some wrapper (e.g. [Taurus](https://gettaurus.org/)), thus it does not contain much features for result processing and input configuration. If you still want to use the tool on the lower level, this section is for you.\n\n### Building from Source\n\nTo build the binary: `go build -o bin/encarno cmd/encarno/main.go`\n\n### Config Format\n\nHere's the full config snippet with some inline comments:\n```yaml\ninput:\n    payloadfile: \"\"      # path to payload input file, mandatory\n    iterationlimit: 0    # if above zero, limits number of times the payload file is looped over\n    stringsfile: \"\"      # if specified, contains string index for payload file\n    enableregexes: false # enables regex related processing\noutput:\n    ldjsonfile: \"\"      # optional, path to results file in LDJSON format\n    reqrespfile: \"\"     # optional, path to detailed trace file\n    reqrespfilelevel: 0 # trace level for the above option\n    binaryfile: \"\"      # optional path to binary results file, also needs strings file if specified\n    stringsfile: \"\"     # for the binary file, the place to write output string index\n    \nworkers:\n    mode: \"\"            # mandatory workload mode, values are 'open' or 'closed'\n    workloadschedule:   # mandatory, the list of linear chunks of workload schedule\n        - levelstart: 0 # starting level for chunk\n          levelend: 10  # ending level for chunk\n          duration: 5s  # duration of chunk\n    startingworkers: 0  # optional, number of initial workers to spawn in open workload \n    maxworkers: 0       # the limit of workers to spawn\n\nprotocol:\n    driver: \"\"        # mandatory, protocol type to use, defaults to 'http', can also be 'dummy' \n    maxconnections: 0 # limit of connections per host in HTTP\n    timeout: 0s       # operation timeout\n    tlsconf:          # TLS custom settings\n        insecureskipverify: false\n        minversion: 0\n        maxversion: 0\n        tlsciphersuites: []\n```\n\n### Payload Input Format\n\nThe format is like that because of possible binary payloads. It starts with single-line JSON of metadata, ending with `\\n`, then `plen` number of bytes, followed by any number of `\\r`, `\\n` or `\\r\\n`.\n\n```text\n{\"plen\": 53, \"address\": \"http://localhost:8070\", \"label\": \"/\"}\nGET / HTTP/1.1\nHost: localhost:8070\nX-Marker: value\n\n\n{\"plen\": 61, \"address\": \"http://localhost:8070\", \"label\": \"/gimme404\"}\nGET /gimme404 HTTP/1.1\nHost: localhost:8070\nX-Marker: value\n\n\n```\n\nThe metadata may contain optional fields for variable evaluation. Below is formatted JSON of metadata for easier understanding:\n```json5\n{\n  \"plen\": 0,     // required, payload length\n  \"label\": \"\",   // item label for grouping in analysis\n  \"address\": \"\", // address for service under test\n  \n  \"replaces\": [\"var1\", \"var2\"], // list of variables to evaluate inside payload\n\n  \"extracts\": {\n    \"varname\": {    // assign the result to this variable name\n      \"re\": \".+\",   // apply this regular expression\n      \"matchNo\": 0, // take this match from results, -1 means random\n      \"groupNo\": 0  // take specific capture group from matched regex\n    }\n  },\n  \"asserts\": [\n    {\n      \"re\": \".+\",     // regex that must exist in response data\n      \"invert\": false // invert the assertion, it would fail if regex is found\n    }\n  ]\n}\n```\n\n\nThe default Taurus configuration would write additional _strings index_ `.istr` file and use `a` and `l` options with string numbers. This is done to minimize the resource footprint. In case you want to see the payload file generated by Taurus without _indexed strings_, use following option:\n```yaml\nmodules:\n  encarno:\n    index-input-strings: false  \n```\n\n\n\n### Results Output Formats\nSpecial code 999 is used for network-level errors.\n\nIt is possible to switch Encarno from default _binary+strings_ format of output file, into single human-readable LDSON file. It is done via special option:\n```yaml\nmodules:\n  encarno:\n    output-format: ldjson  # by default, it's \"bin\"\n```\n\n## History\n\nIt is written as a replacement for the old [phantom](https://github.com/yandex-load/phantom)+[yandex-tank](https://github.com/yandex/yandex-tank) combination.Those were too \"phantom\" (and too unmaintained), we're\ntrying to be \"in flesh\" analogue to it. The idea was to write a tool as precise as phantom, but using modern programming\nlanguage (Go) and address wider spectrum of use-cases.\n\nDuring implementation, it became apparent that some of phantom's concepts are not as important, namely pre-generated\ninput file with schedule and payloads. Also, re-implementing HTTP protocol client was considered as an overkill. Maybe\nwe have lost some speed because of that (we believe not drastically).\n\nIt is intentionally not fully-capable _load testing tool_, it is just _load generator_ that assumes the input preparations and result analysis is done by wrapper scripts.\n\n---\n\n## Changelog\n\n### 0.5.3 -- 29 aug 2022\n- requests from URLs file have default empty label, to avoid garbaging reports with too many labels\n- retry reading string index file\n\n\n### 0.5.1 -- 25 jul 2022\n- regular expression evaluation, extractions and assertions\n- make sure custom payload files work fine\n- add external payload indexed example\n\n### 0.4 -- 21 jun 2022\n- binary output writer\u0026reader, including strings externalization, helper tools to translate into human-readable\n- fixes for ipv6, removed internal round-robin cache for DNS for now\n\n\n### 0.2 and 0.3 -- 13 jun 2022\n* improve automated release process: pypi package, docker image\n\n### 0.1 -- 13 jun 2022\n* add binary releases on GitHub\n* add documentation\n* auto-download binary (needs next release)\n\n### 0.0 -- 10 jun 2022\n* Simple CLI with one config file\n* Open and closed workload support\n* HTTP and dummy protocol types\n* Input file with metadata in JSON line and full payload\n* LDJSON output format\n* Log file with health stats\n* Taurus module with basic scripting\n\n## Roadmap\n\n- respect `iterations` option from Taurus config, test it, handle \"only iterations and no duration is specified\"\n\n- when workers decrease (input exhausted or panics), reflect that in counters\n- unit tests and coverage\n \n- separate file for health status, with per-line flush?\n- fully binary input? is it worth it?\n- global method and body for urls-from-file case?\n- explain log file health KPIs meanings \n- Explain the difference from JMeter and others: How less flexible it is for JMeter, How more flexible it is for Hay and alikes\n- document output file formats\n- document indexed strings format\n\n\n### Parking lot\n\n- auto-USL workload\n- limit len of auto-label for long GET urls\n- udp protocol nib\n- Go plugins used for nib\n\n---\n\n\u003cp align=\"center\"\u003e\u003csup\u003e\u003csub\u003e\u003cimg src=\"https://raw.githubusercontent.com/lipis/flag-icons/main/flags/4x3/pt.svg\" height=\"8\" alt=\"PT\"\u003e Made in Portugal\u003c/sub\u003e\u003c/sup\u003e\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fundera%2Fencarno","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fundera%2Fencarno","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fundera%2Fencarno/lists"}