{"id":13773995,"url":"https://github.com/vy/hrrs","last_synced_at":"2025-04-05T02:08:16.571Z","repository":{"id":17097861,"uuid":"81128415","full_name":"vy/hrrs","owner":"vy","description":"Record, transform, and replay HTTP requests in Java EE and Spring applications.","archived":false,"fork":false,"pushed_at":"2025-02-27T04:17:18.000Z","size":1135,"stargazers_count":83,"open_issues_count":2,"forks_count":15,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-29T01:07:22.547Z","etag":null,"topics":["benchmarking-suite","java","javaee","logger","performance-testing","servlet","spring"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vy.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"COPYING.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"vy"}},"created_at":"2017-02-06T20:14:41.000Z","updated_at":"2025-02-27T04:17:20.000Z","dependencies_parsed_at":"2023-02-17T07:00:28.067Z","dependency_job_id":"954250bb-a97c-4e8b-8da4-b11bda7f43f3","html_url":"https://github.com/vy/hrrs","commit_stats":{"total_commits":337,"total_committers":5,"mean_commits":67.4,"dds":0.4451038575667656,"last_synced_commit":"1b07b93650fded6e2dfef4906b1c3c845f9fbc8c"},"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vy%2Fhrrs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vy%2Fhrrs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vy%2Fhrrs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vy%2Fhrrs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vy","download_url":"https://codeload.github.com/vy/hrrs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247276164,"owners_count":20912288,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarking-suite","java","javaee","logger","performance-testing","servlet","spring"],"created_at":"2024-08-03T17:01:22.750Z","updated_at":"2025-04-05T02:08:16.545Z","avatar_url":"https://github.com/vy.png","language":"Java","readme":"\u003c!---\n Copyright 2016-2024 Volkan Yazıcı\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n        https://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permits and\n limitations under the License.\n--\u003e\n\n[![Actions Status](https://github.com/vy/hrrs/workflows/build/badge.svg)](https://github.com/vy/hrrs/actions)\n[![Maven Central](https://img.shields.io/maven-central/v/com.vlkan.hrrs/hrrs-parent.svg)](https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22com.vlkan.hrrs%22)\n[![License](https://img.shields.io/github/license/vy/hrrs.svg)](https://www.apache.org/licenses/LICENSE-2.0.txt)\n\nHRRS (HTTP Request Record Suite) is a set of tools that you can leverage to\nrecord, transform, and replay HTTP requests in your Java EE and Spring web\napplications written in Java 8 or higher. In essence, HRRS bundles a servlet\nfilter for recording (`hrrs-servlet-filter`) and standalone command-line\nJava applications for transforming (`hrrs-distiller`) and replaying\n(`hrrs-replayer`) the requests.\n\n# Table of Contents\n\n- [Rationale](#rationale)\n- [Overview](#overview)\n- [Getting Started](#getting-started) (setting up a Spring web application,\n  running distiller and replayer)\n- [Recorder Configuration](#recorder-configuration)\n- [Recorder Performance](#recorder-performance)\n- [Replayer Reports](#replayer-reports) (Dropwizard Metrics and JMeter reports)\n- [Distiller \u0026 Replayer Debugging](#debugging)\n- [F.A.Q.](#faq)\n- [Caveats](#caveats)\n- [License](#license)\n\n\u003ca name=\"rationale\"\u003e\u003c/a\u003e\n\n# Rationale\n\nWhy would someone want to record HTTP requests as is? There are two major\nproblems that HRRS is aiming to solve:\n\n- **Realistic performance tests:** Artificially generated test data falls\n  short of covering many production states. Testing with unrealistic user\n  behaviour can cause caches to misbehave. Or benchmarks might have used\n  JSON/XML for simplicity, while the actual production systems communicate\n  over a binary protocol such as Protocol Buffers or Thrift. These short\n  comings undermine the reliability of performance figures and renders\n  regression reports unusable. HRRS lets the production load to be stored\n  and reflected back to the test environment for more credible test results.\n\n- **Diagnosing production problems:** It might not always be a viable option\n  to remotely debug an instance for production surfacing problems. HRRS can be\n  leveraged to record the problem on production and replay it on development\n  environment for further inspection.\n\n- **Warming up standby service caches:** Standby systems are an inevitable\n  part of modern software architectures: reliability, separation of read \u0026\n  write clusters, etc. While replacing primaries with secondary systems, a\n  cold replacement is anticipated to initially yield a degraded performance,\n  which might not be desirable for certain systems. HRRS can be used to warm\n  up the secondaries prior to deployment and alleviate this problem.\n\n\u003ca name=\"overview\"\u003e\u003c/a\u003e\n\n# Overview\n\n![HRRS Overview](doc/overview.png)\n\nHRRS ships the following artifacts:\n\n- **hrrs-api:** Basic API models and interfaces like `HttpRequestHeader`,\n  `HttpRequestRecord`, `HttpRequestRecordReader`,\n  `HttpRequestRecordReaderSource`, etc.\n- **hrrs-servlet-filter:** Basic servlet filter leveraging the functionality\n  of the API interfaces.\n- **hrrs-replayer:** The command line replayer application.\n- **hrrs-distiller:** A command line tool to transform and/or filter stored\n`HttpRequestRecord`s. \n\nThese artifacts provide interfaces for the potential concrete implementations.\nFortunately, we provide one for you: File-based Base64 implementation. That is,\nHTTP request records are encoded in Base64 and stored in a plain text file.\nFollowing artifacts provide this functionality:\n\n- **hrrs-serializer-base64:** The reader/writer implementation using Base64.\n- **hrrs-servlet-filter-base64:** Servlet filter implementation using the Base64\n  serializer.\n- **hrrs-replayer-base64:** The command line replayer implementation using the\n  Base64 serializer.\n- **hrrs-distiller-base64:** The command line distiller implementation using the\n  Base64 serializer.\n\nHRRS is designed with extensibility in mind. As of now, it only supports file\nsourced/targeted Base64 readers/writers. But all you need is a few lines of\ncode to introduce your own serialization schemes powered by a storage backend\n(RDBMS, NoSQL, etc.) of your preference.\n\nSource code also contains the following modules to exemplify the usage of HRRS\nwith certain Java web frameworks:\n\n- **hrrs-example-jaxrs**\n- **hrrs-example-spring**\n\n\u003ca name=\"getting-started\"\u003e\u003c/a\u003e\n\n# Getting Started\n\nIn order to start recording HTTP requests, all you need is to plug the HRRS\nservlet filter into your Java web application. Below, we will use Base64\nserialization for recording HTTP requests in a Spring web application. (See\n`examples` directory for the actual sources and the JAX-RS example.)\n\nAdd the HRRS servlet filter Maven dependency to your `pom.xml`:\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.vlkan.hrrs\u003c/groupId\u003e\n    \u003cartifactId\u003ehrrs-servlet-filter-base64\u003c/artifactId\u003e\n    \u003cversion\u003e${hrrs.version}\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\nIn the second and last step, you expose the HRRS servlet filter as beans so\nthat Spring can inject them as interceptors:\n\n```java\n@Configuration\npublic class HrrsConfig {\n\n    @Bean\n    public HrrsFilter provideHrrsFilter() throws IOException {\n        String tmpPathname = System.getProperty(\"java.io.tmpdir\");\n        String file = new File(tmpPathname, \"hrrs-spring-records.csv\").getAbsolutePath();\n        String filePattern = new File(tmpPathname, \"hrrs-spring-records-%d{yyyyMMdd-HHmmss-SSS}.csv\").getAbsolutePath();\n        RotationConfig rotationConfig = RotationConfig\n                .builder()\n                .file(file)\n                .filePattern(filePattern)\n                .policy(new ByteMatchingRotationPolicy((byte) '\\n', 50_000))\n                .build();\n        return new Base64HrrsFilter(rotationConfig);\n    }\n\n    @Bean\n    public ServletRegistrationBean provideHrrsServlet() {\n        HrrsServlet hrrsServlet = new HrrsServlet();\n        return new ServletRegistrationBean(hrrsServlet, \"/hrrs\");\n    }\n\n}\n```\n\nAnd that's it! The incoming HTTP requests will be recorded into\n`writerTargetFile`. (You can also run `HelloApplication` of `examples/spring`\nin your IDE to see it in action.) All you need to do is instructing the HRRS\nservlet to enable the recorder:\n\n```bash\n$ curl http://localhost:8080/hrrs\n{\"enabled\": false}\n\n$ curl -X PUT http://localhost:8080/hrrs?enabled=true\n```\n\nAfter a couple of `GET /hello?name=\u003cname\u003e` queries, let's take a quick look at\nthe contents of the Base64-serialized HTTP request records:\n\n```bash\n$ zcat records.csv.gz | head -n 3\niz4mjlt9_8f89s  20170213-224106.477+0100  hello  POST  ABYvaGVsbG8/bmFtZT1UZXN0TmFtZS0xAAAABQAEaG9zdAAObG9jYWxob3N0OjgwODAACnVzZXItYWdlbnQAC2N1cmwvNy40Ny4wAAZhY2NlcHQAAyovKgAMY29udGVudC10eXBlAAp0ZXh0L3BsYWluAA5jb250ZW50LWxlbmd0aAACMTMAAAAAAAAAAAAAAA9yYW5kb20tZGF0YS0x//8=\niz4mjlui_1l3bw  20170213-224106.522+0100  hello  POST  ABYvaGVsbG8/bmFtZT1UZXN0TmFtZS0zAAAABQAEaG9zdAAObG9jYWxob3N0OjgwODAACnVzZXItYWdlbnQAC2N1cmwvNy40Ny4wAAZhY2NlcHQAAyovKgAMY29udGVudC10eXBlAAp0ZXh0L3BsYWluAA5jb250ZW50LWxlbmd0aAACMTMAAAAAAAAAAAAAAA9yYW5kb20tZGF0YS0z//8=\niz4mjlty_sicli  20170213-224106.502+0100  hello  POST  ABYvaGVsbG8/bmFtZT1UZXN0TmFtZS0yAAAABQAEaG9zdAAObG9jYWxob3N0OjgwODAACnVzZXItYWdlbnQAC2N1cmwvNy40Ny4wAAZhY2NlcHQAAyovKgAMY29udGVudC10eXBlAAp0ZXh0L3BsYWluAA5jb250ZW50LWxlbmd0aAACMTMAAAAAAAAAAAAAAA9yYW5kb20tZGF0YS0y//8=\n```\n\n(If you can't see any content yet, you can enforce flushing via\n`curl -X POST http://localhost:8080/hrrs`.)\n\nHere each line corresponds to an HTTP request record and fields are separated\nby `\\t` character. A line first starts with plain text id, timestamp, group\nname, and method fields. There it is followed by a Base64-encoded field\ncontaining the URL (including request parameters), headers, and payload. This\nsimple representation makes it easy to employ well-known command line tools\n(`grep`, `sed`, `awk`, etc.) to extract a certain subset of records.\n\n```\n$ zcat records.csv.gz | head -n 1 | awk '{print $5}' | base64 --decode | hd\n00000000  00 16 2f 68 65 6c 6c 6f  3f 6e 61 6d 65 3d 54 65  |../hello?name=Te|\n00000010  73 74 4e 61 6d 65 2d 31  00 00 00 05 00 04 68 6f  |stName-1......ho|\n00000020  73 74 00 0e 6c 6f 63 61  6c 68 6f 73 74 3a 38 30  |st..localhost:80|\n00000030  38 30 00 0a 75 73 65 72  2d 61 67 65 6e 74 00 0b  |80..user-agent..|\n00000040  63 75 72 6c 2f 37 2e 34  37 2e 30 00 06 61 63 63  |curl/7.47.0..acc|\n00000050  65 70 74 00 03 2a 2f 2a  00 0c 63 6f 6e 74 65 6e  |ept..*/*..conten|\n00000060  74 2d 74 79 70 65 00 0a  74 65 78 74 2f 70 6c 61  |t-type..text/pla|\n00000070  69 6e 00 0e 63 6f 6e 74  65 6e 74 2d 6c 65 6e 67  |in..content-leng|\n00000080  74 68 00 02 31 33 00 00  00 00 00 00 00 00 00 00  |th..13..........|\n00000090  00 0f 72 61 6e 64 6f 6d  2d 64 61 74 61 2d 31 ff  |..random-data-1.|\n000000a0  ff                                                |.|\n000000a1\n```\n\nOnce you start recording HTTP requests, you\ncan setup [logrotate](https://github.com/logrotate/logrotate) to periodically\nrotate and compress the record output file. You can even take one step further\nand schedule a cron job to copy these records to a directory accessible by your\ntest environment. There you can replay HTTP request records using the replayer\nprovided by HRRS:\n\n```\n$ java \\\n    -jar /path/to/hrrs-replayer-base64-\u003cversion\u003e.jar \\\n    --targetHost localhost \\\n    --targetPort 8080 \\\n    --threadCount 10 \\\n    --maxRequestCountPerSecond 1000 \\\n    --inputUri file:///path/to/records.csv.gz\n```\n\nBelow is the list of parameters supported by the replayer.\n\n| Parameter | Required | Default | Description |\n| --------- | -------- | ------- | ----------- |\n| `--help`, `-h` | N | false | display this help and exit |\n| `--inputUri`, `-i` | Y | | input URI for HTTP records (Base64 replayer can accept input URIs with `.gz` suffix.) |\n| `--jtlOutputFile`, `-oj` | N | | Apache JMeter JTL output file for test results |\n| `--localAddress`, `-l` | N | | address to bind to when making outgoing connections |\n| `--loggerLevelSpecs`, `-L` | N | `*=warn,com.vlkan.hrrs=info` | comma-separated list of `loggerName=loggerLevel` pairs |\n| `--maxRequestCountPerSecond`, `-r` | N | 1 | number of concurrent requests per second |\n| `--metricsOutputFile`, `-om` | N | | output file to dump Dropwizard metrics |\n| `--metricsOutputPeriodSeconds`, `-mp` | N | 10 | Dropwizard metrics report frequency in seconds |\n| `--rampUpDurationSeconds`, `-d` | N | 1 | ramp up duration in seconds to reach to the maximum number of requests |\n| `--redirectStrategy`, `-rs` | N | `DEFAULT` | redirect strategy (`NONE`, `DEFAULT`, or `LAX`) |\n| `--requestTimeoutSeconds`, `-t` | N | 10 | HTTP request connect/write/read timeout in seconds |\n| `--replayOnce`, `-1` | N | false | exit once all the records are replayed |\n| `--targetHost`, `-th` | Y | | remote HTTP server host |\n| `--targetPort`, `-tp` | Y | | remote HTTP server port |\n| `--threadCount`, `-n` | N | 2 | HTTP request worker pool size |\n| `--totalDurationSeconds`, `-D` | N | 10 | total run duration in seconds |\n\nIt is not always desired to replay recorded HTTP requests as is. One might need\nto exclude certain HTTP headers, remove promotion codes from the URL, sanitize\npayload by shadowing sensitive customer information, etc. You can use distiller\nprovided by HRRS for this purpose:\n\n```\n$ java \\\n    -jar /path/to/hrrs-distiller-base64-\u003cversion\u003e.jar\n    --inputUri file:///path/to/input-records.csv.gz\n    --outputUri file:///path/to/output-records.csv.gz\n    --scriptUri file:///path/to/transform.js\n```\n\nDistiller passes each read input record to the `transform()` function defined\nin the JavaScript file pointed by `--scriptUri` parameter. `transform()`\nreceives a single argument of type `HttpRequestRecord` and returns an\n`HttpRequestRecord`. (Returning `null` lets the distiller to exclude that\nrecord.) Consider the following example:\n\n```javascript\nvar formatter = new java.text.SimpleDateFormat(\"yyyyMMdd-HHmmss.SSSZ\");\nvar loTimestamp = formatter.parse(\"20170415-204551.527+0200\");\nvar hiTimestamp = formatter.parse(\"20170415-204551.700+0200\");\n\n/**\n* Remove `Host` and `Content-Length` headers.\n*/\nfunction sanitizeHeaders(oldHeaders) {\n    var newHeaders = [];\n    for (var oldHeaderIndex = 0; oldHeaderIndex \u003c oldHeaders.length; oldHeaderIndex++) {\n        var oldHeader = oldHeaders[oldHeaderIndex];\n        var oldHeaderName = oldHeader.getName();\n        var allowed =\n            !oldHeaderName.equalsIgnoreCase(\"host\") \u0026\u0026\n            !oldHeaderName.equalsIgnoreCase(\"content-length\");\n        if (allowed) {\n            newHeaders.push(oldHeader);\n        }\n    }\n    return newHeaders;\n}\n\nfunction transform(input) {\n    var timestamp = input.getTimestamp();\n    if (timestamp.after(loTimestamp) \u0026\u0026 timestamp.before(hiTimestamp)) {    // Check the timestamp.\n        var newHeaders = sanitizeHeaders(input.getHeaders());               // Sanitize headers.\n        return input.toBuilder().setHeaders(newHeaders).build();            // Reconstruct record with new headers.\n    }\n    return null;                                                            // Out of time range, ignore the record.\n}\n```\n\nBelow is the list of parameters supported by the distiller.\n\n| Parameter | Required | Default | Description |\n| --------- | -------- | ------- | ----------- |\n| `--help`, `-h` | N | false | display this help and exit |\n| `--inputUri`, `-i` | Y | | input URI for HTTP records |\n| `--loggerLevelSpecs`, `-L` | N | `*=warn,com.vlkan.hrrs=info` | comma-separated list of `loggerName=loggerLevel` pairs |\n| `--outputUri`, `-o` | Y | | output URI for HTTP records |\n| `--scriptUri`, `-s` | Y | | input URI for script file |\n\nFor a more detailed walk-through see [README.md in `examples/spring`](examples/spring/README.md).\n\n\u003ca name=\"recorder-configuration\"\u003e\u003c/a\u003e\n\n# Recorder Configuration\n\nBy default, HRRS servlet filter records every HTTP request along with its\npayload. This certainly is not a desired option for many applications. For such\ncases, you can override certain methods of the `HrrsFilter` to have a more\nfine-grained control over the recorder.\n\n```java\npublic abstract class HrrsFilter implements Filter {\n\n    // ...\n\n    public static final long DEFAULT_MAX_RECORDABLE_PAYLOAD_BYTE_COUNT = 10 * 1024 * 1024;\n\n    /**\n     * Checks if the given HTTP request is recordable.\n     */\n    protected boolean isRequestRecordable(HttpServletRequest request) {\n        return true;\n    }\n\n    /**\n     * Maximum amount of bytes that can be recorded per request.\n     * Defaults to {@link HrrsFilter#DEFAULT_MAX_RECORDABLE_PAYLOAD_BYTE_COUNT}.\n     */\n    public long getMaxRecordablePayloadByteCount() {\n        return DEFAULT_MAX_RECORDABLE_PAYLOAD_BYTE_COUNT;\n    }\n\n    /**\n     * Create a group name for the given request.\n     *\n     * Group names are used to group requests and later on are used\n     * as identifiers while reporting statistics in the replayer.\n     * It is strongly recommended to use group names similar to Java\n     * package names.\n     */\n    protected String createRequestGroupName(HttpServletRequest request) {\n        String requestUri = createRequestUri(request);\n        return requestUri\n                .replaceFirst(\"\\\\?.*\", \"\")      // Replace query parameters.\n                .replaceFirst(\"^/\", \"\")         // Replace the initial slash.\n                .replaceAll(\"/\", \".\");          // Replace all slashes with dots.\n    }\n\n    /**\n     * Creates a unique identifier for the given request.\n     */\n    protected String createRequestId(HttpServletRequest request) {\n        return ID_GENERATOR.next();\n    }\n\n    /**\n     * Filters the given record prior to writing.\n     * @return the modified record or null to exclude the record\n     */\n    protected HttpRequestRecord filterRecord(HttpRequestRecord record) {\n        return record;\n    }\n\n    // ...\n\n}\n```\n\n\u003ca name=\"recorder-performance\"\u003e\u003c/a\u003e\n\n# Recorder Performance\n\nHRRS provided servlet filter wraps the input stream of the HTTP request model.\nWhenever user consumes from the input, we store the read bytes in a seperate\nbuffer, which later on gets Base64-encoded at request completion. There are\ntwo issues with this approach:\n\n- Duplication increases the memory usage.\n- Encoding and storing the requests adds an extra processing overhead.\n\nIt is possible to use a fixed (thread local?) memory pool to avoid extra memory\nallocations for each request. Further, encoding and storing can also be\nperformed in a separate thread to not block the request handler thread.\nThese being said, HRRS is successfully deployed on a 6-node Java EE application\ncluster (each node handles approximately 600 reqs/sec and requests generally\ncontain a payload close to 50KB) without any noticeable memory or processing\noverhead.\n\nAdditionally, you can override `isRequestRecordable()` and\n`getMaxRecordablePayloadByteCount()` methods in `HrrsFilter` to have a more\nfine-grained control over the recorded HTTP requests.\n\n\u003ca name=\"replayer-reports\"\u003e\u003c/a\u003e\n\n# Replayer Reports\n\nIf you have ever used HTTP benchmarking tools like\n[JMeter](https://jmeter.apache.org/) or [Gatling](https://gatling.io/), then\nyou should be familiar with the reports generated by these tools. Rather than\ngenerating its own eye candy reports, HRRS optionally (`--jtlOutputFile`)\ndumps a [JMeter JTL file](https://wiki.apache.org/jmeter/JtlFiles) with the\nstatistics (timestamp, latency, etc.) of each executed request. A quick peek\nat the JMeter JTL file looks as follows:\n\n```xml\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003ctestResults version=\"1.2\"\u003e\n\u003chttpSample t=\"108\" lt=\"108\" ts=\"1486330510795\" s=\"true\" rc=\"200\" lb=\"hello\" tn=\"RateLimitedExecutor-0\"/\u003e\n\u003chttpSample t=\"6\" lt=\"6\" ts=\"1486330510802\" s=\"true\" rc=\"200\" lb=\"hello\" tn=\"RateLimitedExecutor-1\"/\u003e\n\u003chttpSample t=\"3\" lt=\"3\" ts=\"1486330510828\" s=\"true\" rc=\"200\" lb=\"hello\" tn=\"RateLimitedExecutor-0\"/\u003e\n\u003c!-- ... --\u003e\n\u003c/testResults\u003e\n```\n\nFor an overview or to track the progress, you can also command HRRS to\nperiodically dump Dropwizard metrics (`--metricsOutputFile` and\n`--metricsOutputPeriodSeconds`) to a file as well. HRRS uses `ConsoleReporter`\nof Dropwizard metrics to dump the statistics, which look as follows:\n\n```\n__all__\n             count = 10\n         mean rate = 1.01 calls/second\n     1-minute rate = 1.00 calls/second\n     5-minute rate = 1.00 calls/second\n    15-minute rate = 1.00 calls/second\n               min = 3.00 milliseconds\n               max = 108.00 milliseconds\n              mean = 16.15 milliseconds\n            stddev = 29.46 milliseconds\n            median = 8.00 milliseconds\n              75% \u003c= 9.00 milliseconds\n              95% \u003c= 108.00 milliseconds\n              98% \u003c= 108.00 milliseconds\n              99% \u003c= 108.00 milliseconds\n            99.9% \u003c= 108.00 milliseconds\n__all__.200\n             count = 10\n         mean rate = 1.01 calls/second\n     1-minute rate = 1.00 calls/second\n     5-minute rate = 1.00 calls/second\n    15-minute rate = 1.00 calls/second\n               min = 3.00 milliseconds\n               max = 108.00 milliseconds\n              mean = 16.15 milliseconds\n            stddev = 29.46 milliseconds\n            median = 8.00 milliseconds\n              75% \u003c= 9.00 milliseconds\n              95% \u003c= 108.00 milliseconds\n              98% \u003c= 108.00 milliseconds\n              99% \u003c= 108.00 milliseconds\n            99.9% \u003c= 108.00 milliseconds\nhello\n             count = 10\n         mean rate = 1.01 calls/second\n     1-minute rate = 1.00 calls/second\n     5-minute rate = 1.00 calls/second\n    15-minute rate = 1.00 calls/second\n               min = 3.00 milliseconds\n               max = 108.00 milliseconds\n              mean = 16.15 milliseconds\n            stddev = 29.46 milliseconds\n            median = 8.00 milliseconds\n              75% \u003c= 9.00 milliseconds\n              95% \u003c= 108.00 milliseconds\n              98% \u003c= 108.00 milliseconds\n              99% \u003c= 108.00 milliseconds\n            99.9% \u003c= 108.00 milliseconds\nhello.200\n             count = 10\n         mean rate = 1.01 calls/second\n     1-minute rate = 1.00 calls/second\n     5-minute rate = 1.00 calls/second\n    15-minute rate = 1.00 calls/second\n               min = 3.00 milliseconds\n               max = 108.00 milliseconds\n              mean = 16.15 milliseconds\n            stddev = 29.46 milliseconds\n            median = 8.00 milliseconds\n              75% \u003c= 9.00 milliseconds\n              95% \u003c= 108.00 milliseconds\n              98% \u003c= 108.00 milliseconds\n              99% \u003c= 108.00 milliseconds\n            99.9% \u003c= 108.00 milliseconds\n```\n\nHere HRRS updates a Dropwizard timer with label `\u003cgroupName\u003e.\u003cresponseCode\u003e`\nfor each executed request. It also updates the metrics of a pseudo group,\ncalled `__all__`, which covers all the existing groups.\n\n\u003ca name=\"debugging\"\u003e\u003c/a\u003e\n\n# Distiller \u0026 Replayer Debugging\n\nSometimes it becomes handy to have more insight into the distiller and replayer\ninternals. For such cases, you can increase the logging verbosity of certain\npackages. As a starting point, adding `--loggerLevelSpecs \"*=info,com.vlkan.hrrs=trace\"`\nto the command line arguments is generally a good idea. Note that, you don't\nwant to have such a level of verbosity while executing the actual performance\ntests.\n\n\u003ca name=\"faq\"\u003e\u003c/a\u003e\n\n# F.A.Q.\n\n- **What's wrong with JMeter, Gatling, etc.?** There is nothing wrong with\n  them. In fact, they are fantastic tools. I use them on a daily basis for\n  performance tests. Though they do not provide any integration solutions\n  for recording the HTTP traffic of a web application.\n\n- **Then why not just using JMeter, Gatling, etc. as a replayer?** I first\n  started my pursuit by trying to make JMeter replay the HTTP request records\n  that I collected. After wrestling with JMeter and its BeanShell Pre-Processor\n  for days, I implemented a custom fully-fledged replayer using\n  [Apache HTTP Client](http://hc.apache.org/httpcomponents-client-ga/) in a\n  single day. Though it is a lot easier to pull that out using Gatling compared\n  to JMeter. Long story short, I needed JMeter JTL files to integrate my test\n  results in our test infrastructure at work, and a simple replayer did the\n  trick. Though, I welcome any patches for replacing the custom replayer with\n  JMeter and/or Gatling.\n\n- **You could have sniffed HTTP from the raw network traffic.** That would\n  be great! Actually, that is a fantastic idea! Then HRRS would be totally\n  programming language and framework agnostic. Though many Java network packet\n  capturing solutions ([Pcap4j](https://www.pcap4j.org/),\n  [jNetPcap](http://jnetpcap.com/), etc.) require the native\n  [libpcap](http://www.tcpdump.org/) library to be installed on the system.\n  This might be a bold assumption for many deployment environments. Further,\n  deploying a separate executable along with your application might not always\n  be a viable option. As a matter of fact, many deployment environments that I\n  know in the industry still do expect a single JAR/WAR file as a deployable\n  unit. Thus sticking close to the web application itself in Java serves a\n  purpose here.\n\n- **Are you recording the entire HTTP request payload, even if it is not\n  used?** Short answer is *no*. First, the `InputStream` of a request is\n  wrapped and recorded only if `HrrsFilter#isRequestRecordable(HttpServletRequest)`\n  returns `true`. Second, the payload is recorded as long as it is consumed.\n  If the request handler does not consume the payload, then HRRS will not\n  record it either. Additionally, `HrrsFilter#getMaxRecordablePayloadByteCount()`\n  provides a hardcoded upper bound on the maximum number of bytes HRRS is\n  allowed to record. The only exception to this is `x-www-form-urlencoded`\n  requests, see [Caveats](#caveats) section below.\n\n- **Is it possible to query the state of the recorder and enable/disable it at\n  runtime?** Yes, see the usage of `HrrsServlet` above, which provides an HTTP\n  API for that purpose. An MBean exposure is being worked on as well. [TODO]\n\n- **Is using plain text files a good idea for the HTTP records?** Yes and no.\n  Yes, because it suits our needs. It is easier to copy between production and\n  test environments. It is easier to reason about. And you can leverage command\n  line tools (`grep`, `sed`, `awk`, etc.) to manipulate or take a subset of the\n  records. That being said, you can easily implement your own serializers (for\n  instance, using an `RDBMS`) according to your needs.\n\n- **What if the state of the services (e.g., database contents) differ in test\n  and production environments?** We also suffer from the same issue, but that\n  is a totally different problem domain. In our case, the magnitude of the\n  misalignment between production and test environment states are at negligible\n  margins. In the tests, we do expect a stable rate in the HTTP 4XX and 5XX\n  response codes and that works fine for us.\n\n- **Sounds cool! How can I contribute?** Awesome! Just send a pull request\n  over GitHub. In terms of coding conventions, just try to stick to the style\n  in the source code.\n\n\u003ca name=\"caveats\"\u003e\u003c/a\u003e\n\n# Caveats\n\n- **What is up with the `x-www-form-urlencoded` requests?** Long story short,\n  serialization of `x-www-form-urlencoded` requests is an expensive operation\n  and `HrrsFilter#getMaxRecordablePayloadByteCount()` limit is subject to\n  violation.\n\n  In [section SRV.3.1.1 of Servlet spec](https://javaee.github.io/servlet-spec/downloads/servlet-3.1/Final/servlet-3_1-final.pdf),\n  it has been stated that any access to request parameters (e.g. `HttpServletRequest#getParameterMap()`)\n  can trigger the early consumption of the request `InputStream` before it\n  reaches to the handler. If you recall HRRS just wraps the internal\n  `InputStream` to tap the consumed content, this servlet caveat should not\n  constitute a problem. Though parameter parsing methods in Tomcat access the\n  `InputStream` through an internal reference and discard the one that is\n  passed by HRRS. (See [the relevant mailing-list discussion](http://mail-archives.apache.org/mod_mbox/tomcat-users/201709.mbox/browser).)\n  Hence, to be on the safe side, HRRS *re-constructs* the request payload by\n  serializing form parameters back to a byte stream. Additionally, to be\n  able to do that, it also needs to parse and deserialize query parameters\n  and exclude them from the servlet parameters, which is a mixture of both\n  query and form parameters by definition. And unfortunately this nasty\n  operation is relatively more expensive then just cloning an `InputStream`\n  in a regular `POST` request and is not subject to any maximum recordable\n  payload size limits.\n\n- **Which rotation policies can I use?** This completely depends on the\n  serializer you are using. For line-based serializers (e.g., the predefined\n  Base64-based ones) you must use `ByteMatchingRotationPolicy` with `\\n` as\n  the target byte. See #206 for the discussion.\n\n\u003ca name=\"security\"\u003e\u003c/a\u003e\n\n# Security policy\n\nIf you have encountered an unlisted security vulnerability or other unexpected behaviour that has security impact, please report them privately to the [volkan@yazi.ci](mailto:volkan@yazi.ci) email address.\n\n\u003ca name=\"license\"\u003e\u003c/a\u003e\n\n# License\n\nCopyright \u0026copy; 2016-2024 [Volkan Yazıcı](https://volkan.yazi.ci/)\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n   https://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n","funding_links":["https://github.com/sponsors/vy"],"categories":["\u003ca id=\"58b6684347a223e01d4d76d9ca185a88\"\u003e\u003c/a\u003eReplay\u0026\u0026重播"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvy%2Fhrrs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvy%2Fhrrs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvy%2Fhrrs/lists"}