{"id":15131070,"url":"https://github.com/twitter-archive/iago","last_synced_at":"2025-09-28T21:31:00.878Z","repository":{"id":3684493,"uuid":"4754525","full_name":"twitter-archive/iago","owner":"twitter-archive","description":"A load generator, built for engineers","archived":true,"fork":false,"pushed_at":"2020-02-11T16:03:57.000Z","size":1079,"stargazers_count":1348,"open_issues_count":6,"forks_count":142,"subscribers_count":176,"default_branch":"master","last_synced_at":"2024-08-01T08:04:04.951Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://twitter.github.com/iago/","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/twitter-archive.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-06-22T18:44:42.000Z","updated_at":"2024-06-29T20:11:46.000Z","dependencies_parsed_at":"2022-09-06T21:21:19.699Z","dependency_job_id":null,"html_url":"https://github.com/twitter-archive/iago","commit_stats":null,"previous_names":["twitter/iago"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter-archive%2Fiago","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter-archive%2Fiago/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter-archive%2Fiago/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter-archive%2Fiago/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/twitter-archive","download_url":"https://codeload.github.com/twitter-archive/iago/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234563141,"owners_count":18853060,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-26T03:22:36.348Z","updated_at":"2025-09-28T21:30:55.511Z","avatar_url":"https://github.com/twitter-archive.png","language":"Scala","funding_links":[],"categories":["测试"],"sub_categories":[],"readme":"\u003ca name=\"Top\"\u003e\u003c/a\u003e\n\n# Iago, A Load Generator\n[![Build Status](https://secure.travis-ci.org/twitter/iago.png)](http://travis-ci.org/twitter/iago)\n\n* \u003ca href=\"#Iago Quick Start\"\u003eIago Quick Start\u003c/a\u003e\n  - \u003ca href=\"#Iago Prerequisites\"\u003eIago Prerequisites\u003c/a\u003e\n  - \u003ca href=\"#Preparing Your Test\"\u003ePreparing Your Test\u003c/a\u003e\n  - \u003ca href=\"#Executing Your Test\"\u003eExecuting Your Test\u003c/a\u003e\n* \u003ca href=\"#Iago Overview\"\u003eIago Overview\u003c/a\u003e\n  - \u003ca href=\"#Supported Services\"\u003eSupported Services\u003c/a\u003e\n  - \u003ca href=\"#Transaction Requirements\"\u003eTransaction Requirements\u003c/a\u003e\n  - \u003ca href=\"#Sources of Transactions\"\u003eSources of Transactions\u003c/a\u003e\n* \u003ca href=\"#Iago Architecture Overview\"\u003eIago Architecture Overview\u003c/a\u003e\n* \u003ca href=\"#Implementing Your Test\"\u003eImplementing Your Test\u003c/a\u003e\n  - \u003ca href=\"#Scala Example\"\u003eScala Example\u003c/a\u003e\n  - \u003ca href=\"#Scala Thrift Example\"\u003eScala Thrift Example\u003c/a\u003e\n  - \u003ca href=\"#Java Example\"\u003eJava Example\u003c/a\u003e\n  - \u003ca href=\"#Java Thrift Example\"\u003eJava Thrift Example\u003c/a\u003e\n  - \u003ca href=\"#Code Annotations for the Examples\"\u003eCode Annotations for the Examples\u003c/a\u003e\n* \u003ca href=\"#Configuring Your Test\"\u003eConfiguring Your Test\u003c/a\u003e\n\t- \u003ca href=\"#Specifying_Victims\"\u003eSpecifying Victims\u003c/a\u003e\n\t- \u003ca href=\"#extension_point_parameters\"\u003eExtension Point Parameters\u003c/a\u003e\n\t- \u003ca href=\"#sending_large_messages\"\u003eSending Large Messages\u003c/a\u003e\n* \u003ca href=\"#weighted_requests\"\u003eWeighted Requests\u003c/a\u003e\n* \u003ca href=\"#metrics\"\u003eMetrics\u003c/a\u003e\n* \u003ca href=\"#tracing\"\u003eTracing\u003c/a\u003e\n* \u003ca href=\"#artifacts\"\u003eWhat Files Are Created?\u003c/a\u003e\n* \u003ca href=\"#ChangeLog\"\u003eChangeLog\u003c/a\u003e\n* \u003ca href=\"#Contributing\"\u003eContributing to Iago\u003c/a\u003e\n\n\u003ca name=\"Iago Quick Start\"\u003e\u003c/a\u003e\n\n## Iago Quick Start\n\nPlease join [iago-users@googlegroups.com](https://groups.google.com/d/forum/iago-users) for updates and to ask questions.\n\nIf you are already familiar with the Iago Load Generation tool, follow these steps to get started; otherwise, start with the \u003ca href=\"http://twitter.github.com/iago/\"\u003eIago Overview\u003c/a\u003e and perhaps \u003ca href=\"http://twitter.github.com/iago/philosophy.html\"\u003eIago Philosophy\u003c/a\u003e, also known as \"Why Iago?\". For questions, please contact [iago-users@googlegroups.com](https://groups.google.com/d/forum/iago-users).\n\n\u003ca name=\"Iago Prerequisites\"\u003e\u003c/a\u003e\n\n### Iago Prerequisites\n\n1. Download and unpack the Iago distribution.\nWe support Scala 2.10 and recommend you clone the latest master: \u003ca href=\"https://github.com/twitter/iago/zipball/master\"\u003emaster\u003c/a\u003e.\n\n2. Read the documentation.\n\n\u003ca name=\"Preparing Your Test\"\u003e\u003c/a\u003e\n\n### Preparing Your Test\n\n1. Identify your transaction source; see \u003ca href=\"#Transaction Requirements\"\u003eTransaction Requirements\u003c/a\u003e and \u003ca href=\"#Sources of Transactions\"\u003eSources of Transactions\u003c/a\u003e for more information.\n2. In Scala, extend the Iago server's `RecordProcessor` or `ThriftRecordProcessor` class, or in Java, extend `LoadTest` or `ThriftLoadTest`; see \u003ca href=\"#Implementing Your Test\"\u003eImplementing Your Test\u003c/a\u003e for more information.\n3. Create a `launcher.scala` file in your Iago `config` directory with the appropriate settings; see \u003ca href=\"#Configuring Your Test\"\u003eConfiguring Your Test\u003c/a\u003e for more information.\n\n\u003ca name=\"Executing Your Test\"\u003e\u003c/a\u003e\n\n### Executing Your Test\n\nLaunch Iago from the distribution with `java` `-jar` *iago_jar* `-f` *your_config*. This will create the Iago processes for you and configure it to use your transactions. To kill a running job, add `-k` to your launch parameters: `java` `-jar` *iago_jar* `-f` *your_config* `-k`.\n\nIf you launch your Iago job on your local machine and an old Iago job is still running, it probably won't get far: it will attempt to re-use a port and fail. You want to kill the running job, as described above.\n\n\u003cem\u003eIf you build via Maven,\u003c/em\u003e then you might wonder \"How do I launch Iago 'from the distribution'?\" The steps are:\n\u003cpre\u003e\n% \u003ckbd\u003emvn package -DskipTests\u003c/kbd\u003e\n% \u003ckbd\u003emkdir tmp; cd tmp\u003c/kbd\u003e\n% \u003ckbd\u003eunzip ../target/iago-\u003cvar\u003eversion\u003c/var\u003e-package-dist.zip\u003c/kbd\u003e\n% \u003ckbd\u003ejava -jar iago-\u003cvar\u003eversion\u003c/var\u003e.jar -f config/\u003cvar\u003emy_config\u003c/var\u003e.scala\u003c/kbd\u003e\n\u003c/pre\u003e\nDon't assume that you can skip the package/unzip steps if you're just changing a config file. You need to re-package and unzip again.\n\nIf you are using Iago as a library, for example, in the case of testing over the Thrift protocol or building more complex tests with HTTP or Memcached/Kestrel, you should instead add a task to your project's configuration. See \u003ca href=\"#Configuring Your Test\"\u003eConfiguring Your Test\u003c/a\u003e for more information.\n\n[Top](#Top)\n\n\u003ca name=\"Iago Overview\"\u003e\u003c/a\u003e\n\n## Iago Overview\n\nIago is a load generation tool that replays production or synthetic traffic against a given target. Among other things, it differs from other load generation tools in that it attempts to hold constant the transaction rate. For example, if you want to test your service at 100K requests per minute, Iago attempts to achieve that rate.\n\nBecause Iago replays traffic, you must specify the source of the traffic. You use a transaction log as the source of traffic, in which each transaction generates a _request_ to your service that your service processes.\n\nReplaying transactions at a fixed rate enables you to study the behavior of your service under an anticipated load. Iago also allows you to identify bottlenecks or other issues that may not be easily observable in a production environment in which your maximum anticipated load occurs only rarely.\n\n[Top](#Top)\n\n\u003ca name=\"Supported Services\"\u003e\u003c/a\u003e\n\n### Supported Services\n\nIago can generate service requests that travel the net in different ways and are in different formats. The code that does this is in a Transport, a class that extends \u003ccode\u003eParrotTransport\u003c/code\u003e. Iago comes with several Transports already defined. When you configure your test, you will need to set some parameters; to understand which of those parameters are used and how they are used, you probably want to look at the source code for your test's Transport class.\n\n* HTTP: Use \u003ca href=\"https://github.com/twitter/iago/blob/master/src/main/scala/com/twitter/parrot/server/FinagleTransport.scala\"\u003eFinagleTransport\u003c/a\u003e\n* Thrift: Use \u003ca href=\"https://github.com/twitter/iago/blob/master/src/main/scala/com/twitter/parrot/server/ThriftTransport.scala\"\u003eThriftTransport\u003c/a\u003e\n* Memcached: Use \u003ca href=\"https://github.com/twitter/iago/blob/master/src/main/scala/com/twitter/parrot/server/MemcacheTransport.scala\"\u003eMemcacheTransport\u003c/a\u003e\n* Kestrel: Use \u003ca href=\"https://github.com/twitter/iago/blob/master/src/main/scala/com/twitter/parrot/server/KestrelTransport.scala\"\u003eKestrelTransport\u003c/a\u003e\n* UDP: Use \u003ca href=\"https://github.com/twitter/iago/blob/master/src/main/scala/com/twitter/parrot/server/ParrotUdpTransport.scala\"\u003eParrotUdpTransport\u003c/a\u003e\n\nYour service is typically an HTTP or Thrift service written in either Scala or Java.\n\n[Top](#Top)\n\n\u003ca name=\"Transaction Requirements\"\u003e\u003c/a\u003e\n\n### Transaction Requirements\n\nFor replay, Iago recommends you scrub your logs to only include requests which meet the following requirements:\n\n* **Idempotent**, meaning that re-execution of a transaction any number of times yields the same result as the initial execution.\n* **Commutative**, meaning that transaction order is not important. Although transactions are initiated in replay order, Iago's internal behavior may change the actual execution order to guarantee the transaction rate. Also, transactions that implement `Future` responses are executed asynchronously. You can achieve ordering, if required, by using Iago as a library and initiating new requests in response to previous ones. Examples of this are available.\n\n[Top](#Top)\n\n\u003ca name=\"Sources of Transactions\"\u003e\u003c/a\u003e\n\n### Sources of Transactions\n\nTransactions typically come from logs, such as the following:\n\n* Web server logs capture HTTP transactions.\n* Proxy server logs can capture transactions coming through a server. You can place a proxy server in your stack to capture either HTTP or Thrift transactions.\n* Network sniffers can capture transactions as they come across a physical wire. You can program the sniffer to create a log of transactions you identify for capture.\n\nIn some cases, transactions do not exist. For example, transactions for your service may not yet exist because they are part of a new service, or you are obligated not to use transactions that contain sensitive information. In such cases, you can provide _synthetic_ transactions, which are transactions that you create to model the operating environment for your service. When you create synthetic transactions, you must statistically distribute your transactions to match the distribution you expect when your service goes live.\n\n[Top](#Top)\n\n\u003ca name=\"Iago Architecture Overview\"\u003e\u003c/a\u003e\n\n## Iago Architecture Overview\n\nIago consists of _feeders_ and _servers_. A _feeder_ reads your transaction source. A _server_ formats and delivers requests to the service you want to test. The feeder contains a `Poller` object, which is responsible for guaranteeing _cachedSeconds_ worth of transactions in the pipeline to the Iago servers.\n\nMetrics are available in logs and in  graphs as described in [Metrics](#metrics).\n\nThe Iago servers generate requests to your service. Together, all Iago servers generate the specified number of requests per minute. A Iago server's `RecordProcessor` object executes your service and maps the transaction to the format required by your service.\n\nThe feeder polls its servers to see how much data they need to maintain _cachedSeconds_ worth of data. That is how we can have many feeders that need not coordinate with each other.\n\nEnsuring that we go through every last message is important when we are writing traffic summaries in the record processor, especially when the data set is small. The parrot feeder shuts down due to running out of time, running out of data, or both. When the feeder runs out of data we\n\n- make sure that all the data in parrot feeder's internal queues are sent to the parrot server\n- make sure all the data held in the parrot servers cache is sent\n- wait until we get a response for all pending messages or until the reads time out\n\nWhen the parrot feeder runs out of time (the duration configuration) the data in the feeder's internal queues are ignored, otherwise the same process as above occurs.\n\n\n[Top](#Top)\n\n\u003ca name=\"Implementing Your Test\"\u003e\u003c/a\u003e\n\n## Implementing Your Test\n\nThe following sections show examples of implementing your test in both Scala and Java. See \u003ca href=\"#Code Annotations for the Examples\"\u003eCode Annotations for the Examples\u003c/a\u003e for information about either example.\n\n[Top](#Top)\n\n\u003ca name=\"Scala Example\"\u003e\u003c/a\u003e\n\n### Scala Example\n\n\u003cp\u003eTo implement a load test in Scala, you must extend the Iago server's \u003ccode\u003eRecordProcessor\u003c/code\u003e class to specify how to map transactions into the requests that the Iago server delivers to your service. The following example shows a \u003ccode\u003eRecordProcessor\u003c/code\u003e subclass that implements a load test on an \u003ccode\u003eEchoService\u003c/code\u003e HTTP service:\u003c/p\u003e\n\n```scala\npackage com.twitter.example\n\nimport org.apache.thrift.protocol.TBinaryProtocol\n\nimport com.twitter.parrot.processor.RecordProcessor                                     // 1\nimport com.twitter.parrot.thrift.ParrotJob                                              // 2\nimport com.twitter.parrot.server.{ParrotRequest,ParrotService}                          // 3\nimport com.twitter.logging.Logger\nimport org.jboss.netty.handler.codec.http.HttpResponse\n\nimport thrift.EchoService\n\nclass EchoLoadTest(parrotService: ParrotService[ParrotRequest, HttpResponse]) extends RecordProcessor {\n  val client = new EchoService.ServiceToClient(service, new TBinaryProtocol.Factory())  // 4\n  val log = Logger.get(getClass)\n\n  def processLines(job: ParrotJob, lines: Seq[String]) {                                // 5\n    lines map { line =\u003e\n      client.echo(line) respond { rep =\u003e\n        if (rep == \"hello\") {\n          client.echo(\"IT'S TALKING TO US\")                                             // 6\n        }\n        log.info(\"response: \" + rep)                                                    // 7\n      }\n    }\n  }\n}\n```\n\n[Top](#Top)\n\n\u003ca name=\"Scala Thrift Example\"\u003e\u003c/a\u003e\n\n### Scala Thrift Example\n\n\u003cp\u003eTo implement a Thrift load test in Scala, you must extend the Iago server's \u003ccode\u003eThrift RecordProcessor\u003c/code\u003e class to specify how to map transactions into the requests that the Iago server delivers to your service. The following example shows a \u003ccode\u003eThriftRecordProcessor\u003c/code\u003e subclass that implements a load test on an \u003ccode\u003eEchoService\u003c/code\u003e Thrift service:\u003c/p\u003e\n\n```scala\npackage com.twitter.example\n\nimport org.apache.thrift.protocol.TBinaryProtocol\n\nimport com.twitter.parrot.processor.ThriftRecordProcessor                               // 1\nimport com.twitter.parrot.thrift.ParrotJob                                              // 2\nimport com.twitter.parrot.server.{ParrotRequest,ParrotService}                          // 3\nimport com.twitter.logging.Logger\n\nimport thrift.EchoService\n\nclass EchoLoadTest(parrotService: ParrotService[ParrotRequest, Array[Byte]]) extends ThriftRecordProcessor(parrotService) {\n  val client = new EchoService.ServiceToClient(service, new TBinaryProtocol.Factory())  // 4\n  val log = Logger.get(getClass)\n\n  def processLines(job: ParrotJob, lines: Seq[String]) {                                // 5\n    lines map { line =\u003e\n      client.echo(line) respond { rep =\u003e\n        if (rep == \"hello\") {\n          client.echo(\"IT'S TALKING TO US\")                                             // 6\n        }\n        log.info(\"response: \" + rep)                                                    // 7\n      }\n    }\n  }\n}\n```\n\t\t\n[Top](#Top)\n\n\n\u003ca name=\"Java Example\"\u003e\u003c/a\u003e\n\n### Java Example\n\n\u003cp\u003eTo implement a load test in Java, you must extend the Iago server's \u003ccode\u003eLoadTest\u003c/code\u003e class to specify how to map transactions into the requests that the Iago server delivers to your service. The \u003ccode\u003eLoadTest\u003c/code\u003e class provides Java-friendly type mappings for the underlying Scala internals. The following example shows a \u003ccode\u003eLoadTest\u003c/code\u003e subclass that implements a load test on an \u003ccode\u003eEchoService\u003c/code\u003e HTTP service:    \u003c/p\u003e\n\n```java\npackage com.twitter.jexample;\n\nimport com.twitter.example.thrift.EchoService;\nimport com.twitter.parrot.processor.LoadTest;                                           // 1\nimport com.twitter.parrot.thrift.ParrotJob;                                             // 2\nimport com.twitter.parrot.server.ParrotRequest;                                         // 3\n\nimport com.twitter.parrot.server.ParrotService;                                         // 3\nimport com.twitter.util.Future;\nimport com.twitter.util.FutureEventListener;\nimport org.apache.thrift.protocol.TBinaryProtocol;\nimport org.jboss.netty.handler.codec.http.HttpResponse;\n\nimport java.util.List;\n\npublic class EchoLoadTest extends LoadTest {\n  EchoService.ServiceToClient client = null;\n\n  public EchoLoadTest(ParrotService\u003cParrotRequest, HttpResponse\u003e parrotService) {\n    super(parrotService);\n    client = new EchoService.ServiceToClient(service(), new TBinaryProtocol.Factory()); // 4\n  }\n\n  public void processLines(ParrotJob job, List\u003cString\u003e lines) {                         // 5\n    for(String line: lines) {\n      Future\u003cString\u003e future = client.echo(line);\n      future.addEventListener(new FutureEventListener\u003cString\u003e() {\n        public void onSuccess(String msg) {\n          System.out.println(\"response: \" + msg);\n        }\n\n      public void onFailure(Throwable cause) {\n        System.out.println(\"Error: \" + cause);\n      }\n     });\n    }\n  }\n}\n```\n\t\t\n[Top](#Top)\n\n\u003ca name=\"Java Example\"\u003e\u003c/a\u003e\n\n### Java Thrift Example\n\n\u003cp\u003eTo implement a Thrift load test in Java, you must extend the Iago server's \u003ccode\u003eThriftLoadTest\u003c/code\u003e class to specify how to map transactions into the requests that the Iago server delivers to your service. The \u003ccode\u003eThriftLoadTest\u003c/code\u003e class provides Java-friendly type mappings for the underlying Scala internals. The following example shows a \u003ccode\u003eThriftLoadTest\u003c/code\u003e subclass that implements a load test on an \u003ccode\u003eEchoService\u003c/code\u003e Thrift service:    \u003c/p\u003e\n\n```java\npackage com.twitter.jexample;\n\nimport com.twitter.example.thrift.EchoService;\nimport com.twitter.parrot.processor.ThriftLoadTest;                                     // 1\nimport com.twitter.parrot.thrift.ParrotJob;                                             // 2\nimport com.twitter.parrot.server.ParrotRequest;                                         // 3\nimport com.twitter.parrot.server.ParrotService;                                         // 3\nimport com.twitter.util.Future;\nimport com.twitter.util.FutureEventListener;\nimport org.apache.thrift.protocol.TBinaryProtocol;\n\nimport java.util.List;\n\npublic class EchoLoadTest extends ThriftLoadTest {\n  EchoService.ServiceToClient client = null;\n\n  public EchoLoadTest(ParrotService\u003cParrotRequest, byte[]\u003e parrotService) {\n    super(parrotService);\n    client = new EchoService.ServiceToClient(service(), new TBinaryProtocol.Factory()); // 4\n  }\n\n  public void processLines(ParrotJob job, List\u003cString\u003e lines) {                         // 5\n    for(String line: lines) {\n      Future\u003cString\u003e future = client.echo(line);\n      future.addEventListener(new FutureEventListener\u003cString\u003e() {\n        public void onSuccess(String msg) {\n          System.out.println(\"response: \" + msg);\n        }\n\n      public void onFailure(Throwable cause) {\n        System.out.println(\"Error: \" + cause);\n      }\n     });\n    }\n  }\n}\n```\n\t\t\n[Top](#Top)\n\n\u003ca name=\"Code Annotations for the Examples\"\u003e\u003c/a\u003e\n\n### Code Annotations for the Examples\n\nYou define your Iago subclass to execute your service and map transactions to requests for your service:\n\n1. Import `com.twitter.parrot.processor.RecordProcessor` (Scala) or `LoadTest` (Java), whose instance will be executed by a Iago server.\n2. Import `com.twitter.parrot.thrift.ParrotJob`, which contains the Iago server class.\n3. Import `com.twitter.parrot.server.ParrotService` and `com.twitter.parrot.server.ParrotRequest`\n4. Create an instance of your service to be placed under test. Your service is a client of the Iago service.\n5. Define a `processLines` method to format the request and and execute your service.\n6. Optionally, you can initiate a new request based on the response to a previous one.\n7. Optionally, do something with the response. In this example, the response is logged.\n\n[Top](#Top)\n\n\u003ca name=\"Configuring Your Test\"\u003e\u003c/a\u003e\n\n## Configuring Your Test\n\nTo configure your test, create a `launcher.scala` file that that creates a `ParrotLauncherConfig` instance with the configuration parameters you want to set.\n\nThere are several parameters to set. A good one to \u003ca href=\"#Supported Services\"\u003efigure out early is \u003ccode\u003etransport\u003c/code\u003e\u003c/a\u003e; that will in turn help you to find out what, e.g., \u003ccode\u003eresponseType\u003c/code\u003e you need.\n\nThe following example shows parameters for testing a Thrift service:\n\n```scala\nimport com.twitter.parrot.config.ParrotLauncherConfig\n\nnew ParrotLauncherConfig {\n  distDir = \".\"\n  jobName = \"load_echo\"\n  port = 8080\n  victims = \"localhost\"\n  log = \"logs/yesterday.log\"\n  requestRate = 1\n  numInstances = 1\n  duration = 5\n  timeUnit = \"MINUTES\" // affects duration; does not affect requestRate\n\n  imports = \"import com.twitter.example.EchoLoadTest\"\n  responseType = \"Array[Byte]\"\n  transport = \"ThriftTransportFactory(this)\"\n  loadTest = \"new EchoLoadTest(service.get)\"\n}\n```\n\n**Note:** For a sample configuration file, see `config/launcher.scala` within the Iago distribution\u003c/a\u003e.\n\nYou can specify any of the following parameters:\n\n\u003ctable border=\"1\" cellpadding=\"1\"\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eParameter\u003c/th\u003e\n\u003cth\u003eDescription\u003c/th\u003e\n\u003cth\u003eRequired or\u003cbr/\u003eDefault Value\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003ecreateDistribution\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eYou can use this field to create your own distribution rate, instead of having a constant flow. You will need to create a subclass of RequestDistribution and import it.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003cpre\u003ecreateDistribution = \"\"\"createDistribution = {\n  rate =\u003e new MyDistribution(rate)\n}\"\"\"\u003c/pre\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003e\"\"\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003ecustomLogSource\u003c/code\u003e\u003c/td\u003e \u003ctd\u003e\u003cp\u003eA string with Scala code that will be put into the\n    Feeder config. You can use this to get Iago to read in compressed files. Iago can read LZO\n    compressed files using its built-in LzoFileLogSource.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample:\u003c/b\u003e\u003cpre\u003ecustomLogSource = \"\"\"\n  if(inputLog.endsWith(\".lzo\")) {\n    logSource = Some(new com.twitter.parrot.feeder.LzoFileLogSource(inputLog))\n  }\"\"\"\n    \u003c/pre\u003e\u003c/p\u003e\u003c/td\u003e \u003ctd\u003e\u003ci\u003e\"\"\u003c/i\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003edistDir\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eThe subdirectory of your project you're running from, if any.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003edistDir = \"target\"\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003e\".\"\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003edoConfirm\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eIf set to false, you will not be asked to confirm the run.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003edoConfirm = false\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003etrue\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003eduration\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eAn integer value that specifies the time to run the test in \u003ccode\u003etimeUnit\u003c/code\u003e units.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003eduration = 5\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003e\u0026nbsp;\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003efeederXmx\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eDefines feeder heap size. Suggested not to be higher than 4 GB (will cause issues scheduling)\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003efeederXmx = 2048\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003e1744\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003eheader\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eA string value that specifies the HTTP Host header.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003eheader = \"api.yourdomain.com\"\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003e\"\"\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003ehostConnectionCoresize\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eNumber of connections per host that will be kept open, once established, until they hit max idle time or max lifetime\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003ehostConnectionCoresize = 1\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003e1\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003ehostConnectionIdleTimeInMs\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eFor any connection \u003e coreSize, maximum amount of time, in milliseconds, between requests we allow before shutting down the connection\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003ehostConnectionIdleTimeInMs = 50000\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003e60000\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003ehostConnectionLimit\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eLimit on the number of connections per host\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003ehostConnectionLimit = 4\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003eInteger.MAX_VALUE\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003ehostConnectionMaxIdleTimeInMs\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eThe maximum time in milliseconds that any connection (including within core size) can stay idle before shutdown\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003ehostConnectionMaxIdleTimeInMs = 500000\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003e300000\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003ehostConnectionMaxLifeTimeInMs\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eThe maximum time in milliseconds that a connection will be kept open\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003ehostConnectionMaxLifeTimeInMs = 10000\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003eInteger.MAX_VALUE\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n  \u003ctd\u003e\u003ccode\u003ejobName\u003c/code\u003e\u003c/td\u003e\n  \u003ctd\u003e\u003cp\u003eA string value that specifies the the name of your test. This is used for two things:\n      \u003col\u003e\n\t\u003cli\u003eif the parrot feeder is configured to find its servers using zookeeper, and/or \u003c/li\u003e\n\t\u003cli\u003ewhen using mesos it is part of the job names generated. A job name of \"foo\" results in mesos job sharding groups \"parrot_server_foo\" and \"parrot_feeder_foo\".\u003c/li\u003e\n      \u003c/ol\u003e\n    \u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003ejobName = \"testing_tasty_new_feature\"\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003eRequired\u003c/b\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003elocalMode\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eShould Iago attempt to run locally or to use the cluster via mesos?\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003elocalMode = true\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003efalse\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003elog\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eA string value that specifies the complete path to the log you want Iago to replay. If localMode=true then the log should be on your local file system. The log should have at least 1000 items or you should change the \u003ccode\u003ereuseFile\u003c/code\u003e parameter.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003elog = \"logs/yesterday.log\"\u003c/code\u003e\u003c/p\u003e\n    \u003cp\u003e\u003cp\u003eIf localMode=false (the default), then the parrot launcher will copy your log file when attempts to make a package for mesos. You can avoid this, and should, by storing your log file in HDFS.\u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003elog = \"hdfs://hadoop-example.com/yesterday.log\"\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003eRequired\u003c/b\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003eloggers\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eA List of LoggerFactories; allows you to define the type and level of logging you want\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample:\u003c/b\u003e\u003c/p\u003e\n\u003cpre\u003eimport com.twitter.logging.LoggerFactory\nimport com.twitter.logging.config._\n\nnew ParrotLauncherConfig {\n  ...\n  loggers = new LoggerFactory(\n    level = Level.DEBUG,\n    handlers = new ConsoleHandlerConfig()\n  )\n} \u003c/pre\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003eNil\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003emaxRequests\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eAn integer value that specifies the total number of requests to submit to your service.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003emaxRequests = 10000\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003eInteger.MAX_VALUE\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003erequestRate\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eAn integer value that specifies the number of requests per second to submit to your service.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003erequestRate = 10\u003c/code\u003e\u003c/p\u003e\n    \u003cp\u003eNote: if using multiple server instances, requestRate is per-instance, not aggregate.\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003e1\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003ereuseFile\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eA boolean value that specifies whether or not to stop the test when the input log has been read through. Setting this value to true will result in Iago starting back at the beginning of the log when it exhausts the contents. If this is true, your log file should at least be 1,000 lines or more.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003ereuseFile = false\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003etrue\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003escheme\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eA string value that specifies the scheme portion of a URI.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003escheme = \"http\"\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003ehttp\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003eserverXmx\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eDefines server heap size. Suggested not to be higher than 8 GB (will cause issues scheduling)\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003eserverXmx = 5000\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003e4000\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n  \u003ctd\u003e\u003ccode\u003erequestTimeoutInMs\u003c/code\u003e\u003c/td\u003e\n  \u003ctd\u003e\n    \u003cp\u003e(From the Finagle Documentation) The request timeout is the time given to a *single* request (if there are retries, they each get a fresh request timeout). The timeout is applied only after a connection has been acquired. That is: it is applied to the interval between the dispatch of the request and the receipt of the response.\u003c/p\u003e\n    \u003cp\u003eNote that parrot servers will not shut down until every response from every victim has come in. If you've modified your record processor to write test summaries this can be an issue.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003erequestTimeoutInMs = 3000 // if the victim doesn't respond in three seconds, stop waiting\u003c/code\u003e\u003c/p\u003e\n  \u003c/td\u003e\n  \u003ctd\u003e\u003ccode\u003e30000 // 30 seconds\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003ereuseConnections\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eA boolean value that specifies whether connections to your service's hosts can be reused. A value of \u003ccode\u003etrue\u003c/code\u003e enables reuse. Setting this to false greatly increases your use of ephemeral ports and can result in port exhaustion, causing you to achieve a lower rate than requested\u003c/p\u003e\n      \u003cp\u003eThis is only implemented for FinagleTransport.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003ereuseConnections = false\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003etrue\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003ethriftClientId\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eIf you are making Thrift requests, your clientId\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003ethriftClientId = \"projectname.staging\"\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ci\u003e\"\"\u003c/i\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003etimeUnit\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eA string value that specifies time unit of the \u003ccode\u003eduration\u003c/code\u003e. It contains one of the following values:\n        \u003cul\u003e\n            \u003cli\u003e \"MINUTES\"\n            \u003cli\u003e \"HOURS\"\n            \u003cli\u003e \"DAYS\"\n        \u003c/ul\u003e\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003etimeUnit = \"MINUTES\"\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003e\u0026nbsp;\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003etraceLevel\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eA \u003ccode\u003ecom.twitter.logging.Level\u003c/code\u003e subclass. Controls the level of \"debug logging\" for servers and feeders.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample:\u003c/b\u003e\n\u003cpre\u003etraceLevel = com.twitter.logging.Level.TRACE\u003c/pre\u003e\n\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003eLevel.INFO\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003everboseCmd\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eA boolean value that specifies the level of feedback from Iago. A value of \u003ccode\u003etrue\u003c/code\u003e specifies maximum feedback.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003everboseCmd = true\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003efalse\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003ca name=\"Specifying_Victims\"\u003e\u003c/a\u003e\n\n#### [Specifying Victims]\n\nThe point of Iago is to load-test a service. Iago calls these \"victims\".\n\n\nVictims may be a\n\n1. single host:port pair\n2. list of host:port pairs\n3. a zookeeper serverset\n\nNote that ParrotUdpTransport can only handle a single host:port pair. The other transports that come with Iago, being Finagle based, do not have this limitation.\n\n\u003ctable border=\"1\" cellpadding=\"6\"\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eParameter\u003c/th\u003e\n\u003cth\u003eDescription\u003c/th\u003e\n\u003cth\u003eRequired or\u003cbr/\u003eDefault Value\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\n\u003ctr\u003e\n  \u003ctd\u003e\u003ccode\u003evictims\u003c/code\u003e\u003c/td\u003e\n  \u003ctd\u003e\u003cp\u003eA list of host:port pairs:\u003c/p\u003e\n  \u003ccode\u003evictims=\"example.com:80 example2.com:80\"\u003c/code\u003e\n  \u003cp/\u003e\u003cp\u003eA zookeeper server set:\u003c/p\u003e\n  \u003ccode\u003evictims=\"/some/zookeeper/path\"\u003c/code\u003e\n  \u003c/td\u003e\n  \u003ctd\u003e\u003cb\u003eRequired\u003c/b\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003eport\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eAn integer value that specifies the port on which to deliver requests to the \u003ccode\u003evictims\u003c/code\u003e.\u003c/p\u003e\n    \u003cp\u003eThe port is used for two things: to provide a port if none were specified in victims, and to provide a port for the host header using a FinagleTransport.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003eport = 9000\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003eRequired\u003c/b\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003evictimClusterType\u003c/code\u003e\u003c/td\u003e\n  \u003ctd\u003e\n  \u003cp\u003eWhen victimClusterType is \"static\", we set victims and port. victims can be a single host name, a host:port pair, or a list of host:port pairs separated with commas or spaces.\u003c/p\u003e\n  \u003cp\u003eWhen victimClusterType is \"sdzk\" (which stands for \"service discovery zookeeper\") the victim is considered to be a server set, referenced with victims, victimZk, and victimZkPort.\u003c/p\u003e\u003c/td\u003e\n\u003ctd\u003eDefault: \u003ccode\u003e\"static\"\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003evictimZk\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cp\u003ethe host name of the zookeeper where your serverset is registered\u003c/p\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cp\u003eDefault is \u003ccode\u003e\"sdzookeeper.local.twitter.com\"\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n  \u003ctd\u003e\u003ccode\u003evictimZkPort\u003c/code\u003e\u003c/td\u003e\n  \u003ctd\u003e\u003cp\u003eThe port of the zookeeper where your serverset is registered\u003c/p\u003e\u003c/td\u003e\n  \u003ctd\u003e\u003cp\u003eDefault: \u003ccode\u003e2181\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n\u003ca name=\"extension_point_parameters\"\u003e\u003c/a\u003e\n\n#### [Extension Point Parameters]\n\n\u003cp\u003e\u003cstrong\u003eAlternative Use:\u003c/strong\u003e You can specify the following \u003cem\u003eextension point\u003c/em\u003e parameters to configure projects in which Iago is used as both a feeder and server. The Iago feeder provides the log lines to your project, which uses these log lines to form requests that the Iago server then handles:\u003c/p\u003e\n\n\u003ctable border=\"1\" cellpadding=\"6\"\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eParameter\u003c/th\u003e\n\u003cth\u003eDescription\u003c/th\u003e\n\u003cth\u003eRequired or\u003cbr/\u003eDefault Value\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003eimports\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eImports from this project to Iago\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003eIf \u003ccode\u003eProjectX\u003c/code\u003e includes Iago as a dependency, you would specify: \u003cbr/\u003e\n    \u003ccode\u003eimport org.jboss.netty.handler.codec.http.HttpResponse \u003cbr/\u003e\n    import com.twitter.\u003ci\u003eprojectX\u003c/i\u003e.util.ProcessorClass\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003eimport org.jboss.netty.handler.codec.http.HttpResponse\u003cbr/\u003e\n    import com.twitter.parrot.util.LoadTestStub\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003erequestType\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eThe request type of requests from Iago.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExamples:\u003c/b\u003e\n        \u003cul\u003e\n            \u003cli\u003e \u003ccode\u003eParrotRequest\u003c/code\u003e for most services (including HTTP and Thrift)\n        \u003c/ul\u003e \u003c/p\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003eParrotRequest\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003eresponseType\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eThe response type of responses from Iago.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExamples:\u003c/b\u003e\n        \u003cul\u003e\n            \u003cli\u003e \u003ccode\u003eHttpResponse\u003c/code\u003e for an HTTP service\n            \u003cli\u003e \u003ccode\u003eArray[Byte]\u003c/code\u003e for a Thrift service\n        \u003c/ul\u003e \u003c/p\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003eHttpResponse\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003ctd\u003e\u003ccode\u003etransport\u003c/code\u003e\u003c/td\u003e\n  \u003ctd\u003e\n    \u003cp\u003eThe kind of transport to the server, which matches the \u003ccode\u003eresponseType\u003c/code\u003e you want.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample:\u003c/b\u003e\u003ccode\u003etransport = \"ThriftTransportFactory(this)\"\u003c/code\u003e\u003c/p\u003e\n    \u003cp\u003eThe Thrift Transport will send your request and give back \u003ccode\u003eFuture[Array[Byte]]\u003c/code\u003e.\u003c/p\u003e\n  \u003c/td\u003e\n  \u003ctd\u003e\u003ccode\u003eFinagleTransport\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003eloadTest\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cp\u003eYour processor for the Iago feeder's lines, which converts the lines into requests and sends them to the Iago server.\u003c/p\u003e\n    \u003cp\u003e\u003cb\u003eExample: \u003c/b\u003e\u003ccode\u003enew LoadTestStub(service.get)\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ccode\u003enew LoadTestStub(service.get)\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n[Top](#Top)\n\n\u003ca name=\"sending_large_messages\"\u003e\u003c/a\u003e\n\n#### [Sending Large Messages]\n\nBy default, the parrot feeder sends a thousand messages at a time to each connected parrot server until the parrot server has twenty seconds worth of data. This is a good strategy when messages are small (less than a kilobyte). When messages are large, the parrot server will run out of memory. Consider an average message size of 100k, then the feeder will be maintaining an output queue for each connected parrot server of 100 million bytes. For the parrot server, consider a request rate of 2000, then 2000 * 20 * 100k = 4 gigabytes (at least). The following parameters help with large messages:\n\n\u003ctable border=\"1\" cellpadding=\"6\"\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eParameter\u003c/th\u003e\n\u003cth\u003eDescription\u003c/th\u003e\n\u003cth\u003eRequired or\u003cbr/\u003eDefault Value\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctr\u003e\n  \u003ctd\u003e\u003ccode\u003ebatchSize\u003c/code\u003e\u003c/td\u003e\n  \u003ctd\u003e\n    \u003cp\u003ehow many messages the parrot feeder sends at one time to the\n    parrot server. For large messages, setting this to 1 is\n    recommended.\u003c/p\u003e\u003c/td\u003e\n  \u003ctd\u003eDefault: \u003ccode\u003e1000\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003ecachedSeconds\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cp\u003eHow many seconds worth of data the parrot server will attempt to cache. Setting this to 1 for large messages is recommended. The consequence is that, if the parrot feeder garbage-collects, there will be a corresponding pause in traffic to your service unless cachedSeconds is set to a value larger than a typical feeder gc. This author has never observed a feeder gc exceeding a fraction of a second.\u003c/p\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cp\u003eDefault is \u003ccode\u003e20\u003c/code\u003e\u003c/p\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n[Top](#Top)\n\n\u003ca name=\"weighted_requests\"\u003e\u003c/a\u003e\n\n#### [Weighted Requests]\n\nSome applications must make bulk requests to their service. In other words, a single meta-request in the input log may result in several requests being satisfied by the victim. A weight field to ParrotRequest was added so that the RecordProcessor can set and use that weight to control the send rate in the RequestConsumer. For example, a request for 17 messages would be given a weight of 17 which would cause the RequestConsumer to sample the request distribution 17 times yielding a consistent distribution of load on the victim.\n\n[Top](#Top)\n\n\u003ca name=\"metrics\"\u003e\u003c/a\u003e\n\n## [Metrics]\n\nIago uses [Ostrich](https://github.com/twitter/ostrich) to record its metrics. Iago is configured so that a simple graph server is available as long as the parrot server is running. If you are using localMode=true, then the default place for this is\n\n\u0026nbsp;\u0026nbsp;[http://localhost:9994/graph/](http://localhost:9994/graph/)\n\nOne metric of particular interest is\n \n\u0026nbsp;\u0026nbsp;[http://localhost:9994/graph/?g=metric:client/request_latency_ms](http://localhost:9994/graph/?g=metric:client/request_latency_ms)\n\nRequest latency is the time it takes to queue the request for sending until the response is received. See the [Finagle User Guide](http://twitter.github.io/finagle/guide/Metrics.html) for more about the individual metrics.\n\n\nOther metrics of interest:\n\n\u003ctable border=\"1\" cellpadding=\"6\"\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eStatistic\u003c/th\u003e\n\u003cth\u003eDescription\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003econnection_duration\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003eduration of a connection from established to closed?\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003econnection_received_bytes\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003ebytes received per connection\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003econnection_requests\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003eNumber of connection requests that your client did, ie. you can have a pool of 1 connection and the connection can be closed 3 times, so the \"connection_requests\" would be 4 (even if connections = 1)\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003econnection_sent_bytes\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003ebytes send per connection\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003econnections\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003eis the current number of connections between client and server\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003ehandletime_us\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003etime to process the response from the server (ie. execute all the chained map/flatMap)\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003epending\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003eNumber of pending requests (ie. requests without responses)\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003erequest_concurrency\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003eis the current number of connections being processed by finagle\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003erequest_latency_ms\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003ethe time of everything between request/response.\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003e\u003ccode\u003erequest_queue_size\u003c/code\u003e\u003c/td\u003e\n    \u003ctd\u003eNumber of requests waiting to be handled by the server\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003c/table\u003e\n\n\n### [Raggiana]\n\nRaggiana is a simple standalone Finagle stats viewer.\n\nYou can use Raggiana to view the stats log, \u003ca href=\"#artifacts\"\u003eparrot-server-stats.log\u003c/a\u003e, generated by Iago.\n\nYou can clone it from\n\nhttps://github.com/twitter/raggiana\n\nor, just use it directly at\n\nhttp://twitter.github.io/raggiana\n\n[Top](#Top)\n\n\u003ca name=\"tracing\"\u003e\u003c/a\u003e\n\n## [Tracing]\n\nParrot works with [Zipkin](http://twitter.github.io/zipkin/), a distributed tracing system.\n\n[Top](#Top)\n\n\u003ca name=\"artifacts\"\u003e\u003c/a\u003e\n\n## [What Files Are Created?]\n\nThe Iago launcher creates the following files\n\n\tconfig/target/parrot-feeder.scala\n\tconfig/target/parrot-server.scala\n\tscripts/common.sh\n\tscripts/parrot-feeder.sh\n\tscripts/parrot-server.sh\n\nThe Iago feeder creates\n\n\tparrot-feeder.log\n\tgc-feeder.log\n\nThe Iago server creates\n\n\tparrot-server.log\n\tparrot-server-stats.log\n\tgc-server.log \n\nThe logs are rotated by size. Each individual log can be up to 100 megabytes before being rotated. There are 6 rotations maintained.\n\nThe stats log, `parrot-server-stats.log`, is a minute-by-minute dump of all the statistics (or \u003ca\nhref=\"#metrics\"\u003eMetrics\u003c/a\u003e) maintained by the Iago server. Each entry is for the time period since\nthe previous one. That is, all entries in `parrot-server-stats.log` need to be accumulated to match\nthe final values reported by [http://localhost:9994/stats.txt](http://localhost:9994/stats.txt).\n\n[Top](#Top)\n\n## Using Iago as a Library\n\nWhile Iago provides everything you need to target your API with a large distributed loadtest with just a small log processor,\nit also exposes a library of classes for log processing, traffic replay, \u0026 load generation. These can be used in your Iago configuration or incorporated in your application as a library.\n\nparrot/server:\n\n* ParrotRequest: Parrot's internal representation of a request\n* ParrotTransport (FinagleTransport, KestrelTransport, MemcacheTransport, ParrotUdpTransport, ThriftTransport): Interchangeable transport layer for requests to be sent. Parrot contains transport implementations for the following protocols: HTTP (FinagleTransport), Kestrel, Memcache, raw UDP and Thrift.\n* RequestConsumer: Queues ParrotRequests and sends them out on a ParrotTransport at a rate determined by RequestDistribution\n* RequestQueue: A wrapper/control layer for RequestConsumer\n* ParrotService (ParrotThriftService): Enqueues ParrotRequests to a RequestQueue. ParrotThriftService implements finagle's Service interface for use with finagle thrift clients.\n\nparrot/util:\n\n* RequestDistribution: A function specifying the time to arrival of the next request, used to control the request rate. Instances include\n\t* UniformDistribution: Sends requests at a uniform rate\n\t* PoissonProcess: Sends requests at approximatly constant rate randomly varying using a poisson process. This is the default.\n\t* SinusoidalPoissonProcess: Like PoissonProcess but varying the rate sinusoidally.\n\t* SlowStartPoissonProcess: Same as PoissonProcess but starting with a gradual ramp from initial rate to final rate. It will then hold steady at the final rate until time runs out.\n\t* InfiniteRampPoissonProcess: a two staged ramped distribution. Ideal for services that need a warm-up period before ramping up. The rate continues to increase until time runs out.\n\nYou may also find the LogSource and RequestProcessor interfaces discussed earlier useful.\n\nExamples:\n\u003cpre\u003e\n// Make 1000 HTTP requests at a roughly constant rate of 10/sec\n\n// construct the transport and queue\nval client =\n  ClientBuilder()\n    .codec(http())\n    .hosts(\"twitter.com:80\")\n    .build()\nval transport = new FinagleTransport(FinagleService(client))\nval consumer = new RequestConsumer(() =\u003e new PoissionProcess(10)\n// add 1000 requests to the queue\nfor (i \u003c- (1 to 1000)) {\n  consumer.offer(new ParrotRequest(uri= Uri(\"/jack/status/20\", Nil))\n}\n// start sending\ntransport.start()\nconsumer.start()\n// wait for the comsumer to exhaust the queue\nwhile(consumer.size \u003e 0) {\n  Thread.sleep(100)\n}\n// shutdown\nconsumer.shutdown()\ntransport.close()\n\u003c/pre\u003e\n\n\u003cpre\u003e\n// Call a thrift service with a sinusoidally varying rate\n\n// Configure cluster for the service using zookeeper\nval zk = \"zookeeper.example.com\"\nval zkPort = 2181\nval path = \"my/env/role/service\"\nval zookeeperClient = new ZooKeeperClient(Amount.of(1, Time.SECONDS),\n  Seq(InetSocketAddress.createUnresolved(zk, zkPort)).asJava)\nval serverSet = new ServerSetImpl(zookeeperClient, path)\nval cluster = new ZookeeperServerSetCluster(serverSet)\n\n// create transport and queue\nval client =\n  ClientBuilder()\n    .codec(ThriftClientFramedCodec)\n    .cluster(cluster)\n    .build()\nval transport = new ThriftTransport(client)\nval createDistribution = () =\u003e new SinusoidalPoisionProccess(10, 20, 60.seconds)\nval queue = new RequestQueue(new RequestConsumer(createDistribution, transport), transport)\n// create the service and processor\nval service = transport.createService(queue)\nval processor = new EchoLoadTest(service)\n// start sending\ntransport.start()\nconsumer.start()\n// Fill the queue from a logfile\nval source = new LogSourceImpl(\"some_file.txt\")\nwhile (source.hasNext) {\n  processor.processLines(Seq(source.next))\n}\n// wait for the comsumer to exhaust the queue\nwhile(consumer.size \u003e 0) {\n  Thread.sleep(100)\n}\n// shutdown\nconsumer.shutdown()\ntransport.close()\n\u003c/pre\u003e\n\n[Top](#Top)\n\n\u003ca name=\"ChangeLog\"\u003e\u003c/a\u003e\n\n## [ChangeLog]\n\n2013-06-25  release 0.6.7\n\n* graceful shutdown for small log sources\n* dropped vestigial parser config\n* weighted parrot requests\n* supporting large requests (BlobStore): new configurations cachedSeconds \u0026 mesosRamInMb\n* launcher changes: configurable proxy, create config directory if needed, and handle errors better (don't hang)\n* serversets as victims\n* make local logs work with non-local distribution directories\n* kestrel transport transactional get support\n* check generated config files *before* launch\n* LzoFileLogSource for iago\n* Thrift over TLS\n* traceLevel config\n\n[Top](#Top)\n\n\u003ca name=\"Contributing\"\u003e\u003c/a\u003e\n\n## [Contributing to Iago]\n\nIago is open source, hosted on Github \u003ca href=\"http://github.com/twitter/iago\"\u003ehere\u003c/a\u003e.\nIf you have a contribution to make, please fork the repo and submit a pull request.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwitter-archive%2Fiago","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftwitter-archive%2Fiago","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwitter-archive%2Fiago/lists"}