{"id":17104346,"url":"https://github.com/keyang/benman","last_synced_at":"2025-03-23T20:12:13.400Z","repository":{"id":66183428,"uuid":"75197694","full_name":"Keyang/benman","owner":"Keyang","description":null,"archived":false,"fork":false,"pushed_at":"2017-01-06T17:41:26.000Z","size":430,"stargazers_count":1,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-29T03:35:20.820Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Keyang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-11-30T15:01:34.000Z","updated_at":"2016-11-30T16:51:56.000Z","dependencies_parsed_at":"2023-02-26T18:46:00.032Z","dependency_job_id":null,"html_url":"https://github.com/Keyang/benman","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Keyang%2Fbenman","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Keyang%2Fbenman/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Keyang%2Fbenman/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Keyang%2Fbenman/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Keyang","download_url":"https://codeload.github.com/Keyang/benman/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245162194,"owners_count":20570692,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T15:36:27.349Z","updated_at":"2025-03-23T20:12:13.372Z","avatar_url":"https://github.com/Keyang.png","language":"JavaScript","readme":"#Benman\nBenchmark / load testing framework based on Postman collections.\n\n#Quick Start\n\n##Installation\n\n```\nnpm install -g benman \n```\n\n##Run postman collections\n\n```\nbenman -p \u003cPath to collection file\u003e\n```\n\nSimple as it is.\n\n\n#Advanced Usage\n\n## Use from Command Line \n\n`benman` takes several parameters. You can find them by running `benman --help`:\n\n```\n$benman --help\n\n  Usage: benman \u003c-b | -p\u003e [options] \u003cpath\u003e\n\n  Benchmark / load testing endpoints with Postman collections.\n\n  Options:\n\n    -h, --help                 output usage information\n    -V, --version              output the version number\n    -b, --benman               The benman config to run\n    -p, --postman              The postman config to run\n    -u, --usernumber \u003cnumber\u003e  The number of concurrent simulated users. Default: 1\n    -l, --loopnumber \u003cnumber\u003e  Number of execution for each simulated user. Default: 5\n    -r, --rampup \u003csecond\u003e      Ramp up period for simulated user. Default: 0\n    -a, --aggregator \u003cname\u003e    Name of result aggregator to use. Default: timeElapse\n    --agent-url \u003cagentUrl\u003e     Optional. the agent url connect to. If omitted, a local agent will be spawned.\n```\n\n\n## Use as library in Node.js app\n\n```js\nvar benman=require(\"benman\");\n/**\nbenman.Agent //agent defines where to run load testing script. It can be local or remote\nbenman.Benman //Benman instance contains definition of all load testing script units / configurations . the instance also provides methods.\nbenman.Unit // A unit is a wrapper of postman collection to add fields like number of concurrent users / iteration count / ramp up period etc.\n*/\n\nbenman.Agent.createLocalAgent({\n\tworkerNum:4 //create a local agent with 4 child processes (workers)\n})\n.then(function(agent){\n\treturn agent.connect() //connect to agent\n\t.then(function(){\n\t\treturn agent.getStatus() //check agent status\n\t})\n\t.then(function(status){\n\t\tif (status.status ===\"ready\"){ //agent is ready for work\n\t\t\treturn agent.run(myBenmanJSON); // run benman json configuration\n\t\t}else{\n\t\t\t//handle status.message\n\t\t}\n\t})\n});\n```\n\n\n## Start from Postman\nIn Postman, create basic HTTP requests.\n\nCreate Postman Collection and add all the required HTTP requests.\n\nStart a Collection Runner in Postman and run the collection to confirm all HTTP requests run as expected.\n\nExport the Collection as a `Collection v2` json file.  This is the Postman config file which should be specifed using the `-p` file when running `benman`\n\n\n\n \n\n# Agents\n\nAgents may be used to run test collections distributed across multiple machines.  Run  `benman-agent` on each machine to set it up as an Agent.\n\nRun tests on the Agent machine by specifying the host ip address and port number using the `--agent-url` option in the `benman` command\n\n```\nbenman --agent-url \u003cip address\u003e:\u003cport no.\u003e -p \u003cname\u003e.postman_collection.json\n```\n\n\n## Use from Command Line \nSet up Agents using `benman-agent` from the command line\n\n`benman-agent` takes several parameters. You can find them by running `benman-agent --help`:\n\n\n```\nbenman-agent --help\n\n  Usage: benman-agent [options]\n\n  Benman remote agent. Used with benman to perform remote load testing scripts.\n\n  Options:\n\n    -h, --help                   output usage information\n    -V, --version                output the version number\n    -h, --host \u003chost\u003e            Network interface to listen on. Default: 0.0.0.0\n    -p, --port \u003cport\u003e            The port to listen on. Default: 9901\n    -w, --workerNum \u003cworkerNum\u003e  Number of local worker number. Default: 2\n    \n```\n\nOnce `benman-agent` is running, run test Collection in Command Line terminal using\n\n```\nbenman --agent-url 0.0.0.0:9901 -p \u003cname\u003e.postman_collection.json\n```\n\n\n# Aggregators\n\nAggregators parse and report on results returned from the test run.\n\nAggregators can be installed using NPM or you can write your own and save in the ***aggregators*** folder.  See section on ***Creating Aggregators*** below\n\nThe default aggregator, stored in the ***aggregators*** folder, is `timeElapse.js`. \n\n\n```\n/**\n * Time elapse aggregator will report following:\n * minTime  -- minimum time elapse for running the collection\n * maxTime -- maximum time elapse for running the collection\n * avgTime -- average time elpase for running the collection\n * timeArr -- array of all time elapsed\n * numberOfFailures -- count of request failures\n * numberOfRequests -- number of total requests made\n */\n```\n\nExample default output for a collection with 4 HTTP requests is:\n\n```\n[\n  {\n    \"name\": \"benman\",\n    \"timeElapse\": {\n      \"minTime\": 1522,\n      \"maxTime\": 3300,\n      \"timeArr\": [\n        1568,\n        1522,\n        3192,\n        3300,\n        3270\n      ],\n      \"avgTime\": 2570.4,\n      \"numberOfFailures\": 0,\n      \"numberOfRequests\": 20\n    }\n  }\n]\n```\n\n# Creating Aggregators\n\nTemplate File for creating your own Aggregator is as follows:\n\n```js\nvar Aggregator = require(\"./Aggregator\");\nvar util = require(\"util\");\n\nfunction customAggregatorName() {\n  Aggregator.call(this);\n}\nutil.inherits(TimeElapse, Aggregator);\n\ncustomAggregatorName.prototype.mapEach = function (item) {\n  // ***** add code here *****\n}\n\ncustomAggregatorName.prototype.reduceEach = function (item, lastResult) {\n  // ***** add code here *****\n}\n\nmodule.exports = new TimeElapse();\n```\n\nWrite your won `mapEach` and `reduceEach` functions to summarise the results returned from the test run according to your requirements.\n\nAn example of the format of the `item` object returned from each test run is as follows where `summary` will be an array with an entry for each HTTP request in the Collection.\n\n```\n{\n  \"hasError\": false,\n  \"summary\": [\n    {\n      \"error\": null,\n      \"summary\": {\n        \"responseCode\": 200,\n        \"responseBodySize\": 20,\n        \"responseHeaders\": {\n          \"access-control-allow-origin\": \"*\",\n          \"content-type\": \"application/json\",\n          \"date\": \"Mon, 19 Dec 2016 10:40:04 GMT\",\n          \"etag\": \"\\\"1773269663\\\"\",\n          \"x-powered-by\": \"Express\",\n          \"content-length\": \"20\",\n          \"connection\": \"Close\"\n        }\n      },\n      \"start\": 1482144004247,\n      \"end\": 1482144004758,\n      \"responseTime\": 511\n    }\n  ],\n  \"totalResponseTime\": 2222,\n  \"endTime\": 1482144006479,\n  \"startTime\": 1482144001325\n}\n```\n\nFor reference, examples of `mapEach` and `reduceEach` functions from the default `timeElapse` aggregator are as follows: \n\n```\nTimeElapse.prototype.mapEach = function (item) {\n  var _ = this._;\n  return {\n    responseTime: _.get(item, \"totalResponseTime\"),\n    numberOfFailures: _.filter(item.summary, function (sum) {\n      return !!sum.error;\n    }).length,\n    numberOfRequests: item.summary.length\n  }\n}\n\nTimeElapse.prototype.reduceEach = function (item, lastResult) {\n  if (!lastResult) {\n    lastResult = {\n      minTime: Number.POSITIVE_INFINITY,\n      maxTime: Number.NEGATIVE_INFINITY,\n      avgTime: 0,\n      timeArr: [],\n      numberOfFailures: 0,\n      numberOfRequests: 0\n    };\n  }\n  return {\n    minTime: Math.min(lastResult.minTime, item.responseTime),\n    maxTime: Math.max(lastResult.maxTime, item.responseTime),\n    timeArr: lastResult.timeArr.push(item.responseTime) \u0026\u0026 lastResult.timeArr,\n    avgTime: lastResult.timeArr.reduce(function (a, b) { return a + b }, 0) / lastResult.timeArr.length,\n    numberOfFailures: lastResult.numberOfFailures += item.numberOfFailures,\n    numberOfRequests: lastResult.numberOfRequests += item.numberOfRequests\n  };\n}\n```\n\n\n\n# Examples\n\n\nIn Postman, create basic HTTP requests.\n\n![Alt text](./images/bGetReqParams.png?raw=true \"HTTP request\")\n\nCreate Postman Collection and add all the required HTTP requests.\n\n### Export the Collection from Postman\n\n![Alt text](./images/bCollectionCreate.png?raw=true \"Create Collection\")\n\n![Alt text](./images/bCollectionSave.png?raw=true \"Create Collection\")\n\n\n### Export as a Collection v2 file\n\n![Alt text](./images/bCollectionExport.png?raw=true \"Export Collection\")\n\nThis will generate a Postman Collection config json file similar to:\n\n```\n{\n\t\"variables\": [],\n\t\"info\": {\n\t\t\"name\": \"benman\",\n\t\t\"_postman_id\": \"6ba8a8bf-5c8f-56ec-a4e9-58c8a3a0c0d3\",\n\t\t\"description\": \"\",\n\t\t\"schema\": \"https://schema.getpostman.com/json/collection/v2.0.0/collection.json\"\n\t},\n\t\"item\": [\n\t\t{\n\t\t\t\"name\": \"get with params\",\n\t\t\t\"event\": [\n\t\t\t\t{\n\t\t\t\t\t\"listen\": \"test\",\n\t\t\t\t\t\"script\": {\n\t\t\t\t\t\t\"type\": \"text/javascript\",\n\t\t\t\t\t\t\"exec\": [\n\t\t\t\t\t\t\t\"\",\n\t\t\t\t\t\t\t\"tests[\\\"Status code is 200\\\"] = responseCode.code === 200;\",\n\t\t\t\t\t\t\t\"var jsonData = JSON.parse(responseBody);\",\n\t\t\t\t\t\t\t\"tests[\\\"name returned is Mary\\\"] = jsonData.msg === 'Hello Mary'\",\n\t\t\t\t\t\t\t\"\",\n\t\t\t\t\t\t\t\"tests[\\\"Response time is less than 200ms\\\"] = responseTime \u003c 2000;\",\n\t\t\t\t\t\t\t\"\",\n\t\t\t\t\t\t\t\"var schema = {\",\n\t\t\t\t\t\t\t\" \\\"type\\\": \\\"object\\\",\",\n\t\t\t\t\t\t\t\" \\\"required\\\":[\\\"msg\\\"],\",\n\t\t\t\t\t\t\t\" \\\"properties\\\" : {\",\n\t\t\t\t\t\t\t\"        \\\"msg\\\" : { \\\"type\\\" : \\\"string\\\" } \",\n\t\t\t\t\t\t\t\"    }\",\n\t\t\t\t\t\t\t\"};\",\n\t\t\t\t\t\t\t\"tests[\\\"Valid Data1\\\"] = tv4.validate(jsonData, schema);\",\n\t\t\t\t\t\t\t\"\",\n\t\t\t\t\t\t\t\"\"\n\t\t\t\t\t\t]\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t],\n\t\t\t\"request\": {\n\t\t\t\t\"url\": \"https://psdev-lpsrnkenozdzjvfx7xflazzl-evals-dev.mbaas1.tom.redhatmobile.com/hello?hello=Mary\",\n\t\t\t\t\"method\": \"GET\",\n\t\t\t\t\"header\": [\n\t\t\t\t\t{\n\t\t\t\t\t\t\"key\": \"Content-Type\",\n\t\t\t\t\t\t\"value\": \"application/json\",\n\t\t\t\t\t\t\"description\": \"\"\n\t\t\t\t\t}\n\t\t\t\t],\n\t\t\t\t\"body\": {\n\t\t\t\t\t\"mode\": \"raw\",\n\t\t\t\t\t\"raw\": \"\"\n\t\t\t\t},\n\t\t\t\t\"description\": \"\"\n\t\t\t},\n\t\t\t\"response\": []\n\t\t}\n\t]\n}\n```\n\n\n### Run Benman from command line using \n\n```\nbenman -p benman.postman_collection.json\n```\n\n### Example default output for a collection with 4 HTTP requests is:\n\n```\n[\n  {\n    \"name\": \"benman\",\n    \"timeElapse\": {\n      \"minTime\": 1522,\n      \"maxTime\": 3300,\n      \"timeArr\": [\n        1568,\n        1522,\n        3192,\n        3300,\n        3270\n      ],\n      \"avgTime\": 2570.4,\n      \"numberOfFailures\": 0,\n      \"numberOfRequests\": 20\n    }\n  }\n]\n```\n\n\n#License\nMIT\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkeyang%2Fbenman","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkeyang%2Fbenman","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkeyang%2Fbenman/lists"}