{"id":26032730,"url":"https://github.com/apache/geode-benchmarks","last_synced_at":"2025-10-03T13:53:52.237Z","repository":{"id":37926040,"uuid":"158306420","full_name":"apache/geode-benchmarks","owner":"apache","description":"Apache Geode Benchmarks","archived":false,"fork":false,"pushed_at":"2023-01-23T05:07:28.000Z","size":1837,"stargazers_count":5,"open_issues_count":2,"forks_count":27,"subscribers_count":46,"default_branch":"develop","last_synced_at":"2025-04-03T04:16:55.744Z","etag":null,"topics":["geode"],"latest_commit_sha":null,"homepage":"https://geode.apache.org/","language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apache.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-20T00:09:28.000Z","updated_at":"2022-06-28T19:58:11.000Z","dependencies_parsed_at":"2023-02-12T20:45:15.063Z","dependency_job_id":null,"html_url":"https://github.com/apache/geode-benchmarks","commit_stats":null,"previous_names":[],"tags_count":69,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fgeode-benchmarks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fgeode-benchmarks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fgeode-benchmarks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fgeode-benchmarks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apache","download_url":"https://codeload.github.com/apache/geode-benchmarks/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247909203,"owners_count":21016479,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["geode"],"created_at":"2025-03-06T22:56:54.144Z","updated_at":"2025-10-03T13:53:47.204Z","avatar_url":"https://github.com/apache.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://travis-ci.org/apache/geode-benchmarks.svg?branch=develop)](https://travis-ci.org/apache/geode-benchmarks)\n[![Total alerts](https://img.shields.io/lgtm/alerts/g/apache/geode-benchmarks.svg?logo=lgtm\u0026logoWidth=18)](https://lgtm.com/projects/g/apache/geode-benchmarks/alerts/)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0)\n\n\n# Apache Geode Benchmarks\n\nThis project contains a benchmarking framework and benchmarks for \n[Apache Geode](https://geode.apache.org/). It is based on the \n[yardstick framework](https://github.com/gridgain/yardstick), but with a java based\ntest configuration and test execution framework.\n\nThese benchmarks are under development.\n\n## Running the benchmarks\nThe benchmarks require machines with passwordless ssh enabled in order to run, so ensure that the authentication key pair for SSH does not have a passphrase. If you have previously created a key pair with a passphrase, you can create a different key pair which\nis of a different type than the previously created one. Be sure to backup your existing key pair before creating a new one. The public key needs to be in PEM format, but some newer OpenSSH\nversions default to a new format. PEM format can be forced by using `-m PEM`:\n```\nssh-keygen -m PEM -t rsa\n```\nWhile running a test on a single machine (i.e. localhost) add the generated key to `authorized_keys` to authorize the user:\n```\ncat \u003cyour_public_key_file\u003e \u003e\u003e ~/.ssh/authorized_keys\n```\nTest if you can ssh to localhost:\n```\nssh localhost\n```\nAs long as that works, we are good to go.\n\nGet your local hosts name:\n```\nhostname\n```\nEdit /etc/hosts and add the local host name with and without domain to localhost entries.\n```\n127.0.0.1 localhost mycomputer mycomputer.mydomain\n::1       localhost mycomputer mycomputer.mydomain\n```\n \nTo run all benchmarks, run the benchmark task and pass in a list of hosts.\n\nFor example:\n```\n./gradlew benchmark -Phosts=localhost,localhost,localhost,localhost\n```\n\n### Options\nThe benchmarks can take configuration options. Some using Gradle's `-P` flag and other, which adjust\nbenchmark behavior, via Java system properties using `-D`.\n\n| Option            | Description                                                                                                                                                                                                                     |\n|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `-Phosts`         | Hosts used by benchmarks on the order of client,locator,server,server (-Phosts=localhost,localhost,localhost,localhost)                                                                                                         |\n| `-PoutputDir`     | Results output directory (-PoutputDir=/tmp/results)                                                                                                                                                                             |\n| `-Pbenchmark.X`   | Where X is a benchmark configuration, defined below.                                                                                                                                                                            |\n| `-Pbenchmark.R.X` | Applies properties to specific roles. Where R is the role, \"client\", \"server\", \"locator\", X is a benchmark configuration, defined below.                                                                                        |\n| `--tests`         | Specific benchmarks to run (--tests=PartitionedPutBenchmark)                                                                                                                                                                    |\n| `-d`              | Debug                                                                                                                                                                                                                           |\n| `-i`              | Info                                                                                                                                                                                                                            |\n| ~~`-PtestJVM`~~   | (Deprecated, use `-Pbenchmark.withToolchainVersion`) ~~Path to an alternative JVM for running the client, locator, and servers. If not specified JAVA_HOME will be used. Note all compilation tasks will still use JAVA_HOME.~~ |\n\n#### Benchmark Configuration\n##### Common\nThese options may apply to all benchmarks.\n\n| Option               | Description                                                                                                                      |\n|----------------------|----------------------------------------------------------------------------------------------------------------------------------|\n| withToolchainVersion | Version number of alternative JVM for running benchmarks, 8 (default), 11, 17.                                                   |\n| withGc               | Select which GC to use. Valid values CMS (default), G1, Z, Shenandoah, Epsilon. (optionally per-role)                            |\n| withHeap             | Specify how large a heap the benchmark VMs should use, default \"8g\". Accepts any `-Xmx` value, like \"32g\". (optionally per-role) |\n| withThreads          | Specify how many threads to use when executing the benchmark. Default varies by benchmark.                                       |\n| withWarmup           | Specify how long to warm up the benchmark in seconds. Default is 60 seconds.                                                     |\n| withDuration         | Specify how long to run the benchmark in seconds. Default is 300 seconds.                                                        |\n| withMinKey           | The minimum key value in the key range. Default is 0.                                                                            |\n| withMaxKey           | The maximum key value in the key range. Default varies by benchmark.                                                             |\n| withLocatorCount     | Number of locators a topology should use. Typically defaults to 1.                                                               |\n| withServerCount      | Number of servers a topology should use. Typically defaults to 2.                                                                |\n| withClientCount      | Number of clients a topology should use. Typically defaults to 1.                                                                |\n| withReplicas         | Number of region replicas.                                                                                                       |\n| withAsyncReplication | Enable asynch region replication.                                                                                                |\n| withNettyThreads     | Number of threads Netty IO Services should have.                                                                                 |\n\n##### Geode Benchmarks\nThese options only apply to Geode benchmarks.\n\n| Option              | Description                                                                         |\n|---------------------|-------------------------------------------------------------------------------------|\n| withSsl             | Flag to run geode with SSL. A self-signed certificate will be generated at runtime. |\n| withSslProtocols    | Specifies enabled SSL protocols. See Geode property `ssl-protocols`                 |\n| withSslCiphers      | Specifies enabled SSL chipher suites. See Geode property `ssl-ciphers`              |\n| withSecurityManager | Flag to start Geode with the example implementation of SecurityManager              |\n| withSniProxy        | Use SNI proxy topology.                                                             |\n| withSniProxyImage   | Provide an alternative Docker image coordinate for SNI proxy.                       |\n| withRouter          | Use router with SNI proxy topology.                                                 |\n| withRouterImage     | Provide an alternative Docker image coordinate for router.                          |\n\n##### Debugging\nThese options should not be used when measuring benchmarks.\n\n| Option               | Description                                                              |\n|----------------------|--------------------------------------------------------------------------|\n| withValidation       | Enable validation of operations. Default disabled.                       |\n| withGcLogging        | Enable GC logging. Default disabled.                                     |\n| withSafepointLogging | Enable Safepoint logging. Default disabled.                              |\n| withStrace           | Launch remote JVM via strace for tracing system calls. Default disabled. |\n\n\n### Scripts for running in aws and analyzing results\n\nThis project includes some scripts to automate running benchmarks in AWS and analyzing the results produced (as well as the results produced from running locally). See the \n[README.md](infrastructure/scripts/aws/README.md) in the infrastructure/aws directory.\n\n## Project structure\n\nThe project is divided into two modules\n* harness - Contains test framework code for running benchmarks. Does not depend on Apache Geode.\n* geode-benchmarks - Individual benchmarks of Apache Geode.\n\n## Sample benchmark\n\nBenchmarks are defined in declarative configuration classes. Each configuration class is run as \na junit test which calls the configure method and passes it to the TestRunner, which executes\nthe test on the provided TEST_HOSTS.\n\nBenchmarks are composed of `before` tasks, `after` tasks, and `workload` tasks. Each seperate `before` and `after`\ntask is run once. `Workload` tasks are run repeatedly and their execution time is measured and\nreported by the yardstick framework. \n\n```java\n/**\n* Benchmark configuration class, which defines the topology of the test and\n* the initialization tasks and workload tasks for the test.\n*/\npublic class PartitionedPutBenchmark extends AbstractPerformanceTest {\n\n  @Test\n  public void run() throws Exception {\n    TestRunners.defaultRunner().runTest(this::configure);\n  }\n  \n  /**\n  * Declare the configuration of the test by calling methods\n  * on TestConfig.\n  */\n  public TestConfig configure() {\n    TestConfig testConfig = new TestConfig();\n    int locatorPort = 10334;\n\n    //This test has three roles, a geode locator, server, and client\n    config.role(\"locator\", 1);\n    config.role(\"server\", 2);\n    config.role(\"client\", 1);\n    \n    //Define how the locator,server and client are initialized\n    config.before(new StartLocator(locatorPort), \"locator\");\n    config.before(new StartServer(locatorPort), \"server\");\n    config.before(new StartClient(locatorPort), \"client\");\n    config.before(new CreatePartitionedRegion(), \"server\");\n    config.before(new CreateClientProxyRegion(), \"client\");\n    //Define the benchmarked workload, which runs in a client\n    config.workload(new PutTask());\n    \n    return config;\n  }\n}\n```\n\n```java\n/**\n* Workload task, which extends the yardstick BenchmarkDriverAdapter\n* \n* Workload tasks should execute a single unit of work, and will be run repeatedly\n* for the duration of the test.\n*/\npublic class PutTask extends BenchmarkDriverAdapter implements Serializable {\n  private Region\u003cObject, Object\u003e region;\n  \n  @Override\n  public void setUp(BenchmarkConfiguration cfg) throws Exception {\n    super.setUp(cfg);\n    ClientCache cache = ClientCacheFactory.getAnyInstance();\n    region = cache.getRegion(\"region\");\n  }\n\n  \n  @Override\n  public boolean test(Map\u003cObject, Object\u003e ctx) throws Exception {\n    region.put(1,2);\n    return true;\n  }\n}\n```\n\n## SNI Proxy\n\nYou can run any benchmark on a topology that routes all client-server communication through an SNI proxy.\n\nThe `withSniProxy` property accepts:\n * `HAProxy` for HAProxy based SNI proxy (default).\n * `Envoy` for Envoy based SNI proxy.\n * `Manual` for providing your own SNI proxy and managing its lifecycle.\n \nThe `withSniProxyImage` property can be used to provide an alternative Docker image to one of the \nsupported proxy implementations. The value should be set to a valid Docker image coordinate. \n \nTo run a test, e.g. `PartitionedGetBenchmark`, with default SNI Proxy:\n```console\n./run_tests.sh -t anytagname -- -Pbenchmark.withSniProxy --tests=PartitionedGetBenchmark\n```\n\nSince SNI is a feature of TLS, running with the SNI topology incurs TLS overheads with implied `-Pbenchmark.withSsl`.\n\n### Router\nAn alternative topology uses a router sitting in front of the SNI proxy to simulate off network access\nto the cluster, enabled with `-Pbenchmark.withRouter`.\n\nEnabling the router implies `-Pbenchmark.withSniProxy`.\n\nThe `withRouter` property accepts:\n * `HAProxy` for HAProxy based router (default).\n * `Manual` for providing your own router and managing its lifecycle.\n\nExample:\n```console\n./run_tests.sh -t anytagname -- -Pbenchmark.withRouter --tests=PartitionedGetBenchmark\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Fgeode-benchmarks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapache%2Fgeode-benchmarks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Fgeode-benchmarks/lists"}