{"id":19914746,"url":"https://github.com/fullstorydev/solr-bench","last_synced_at":"2025-05-03T05:31:51.555Z","repository":{"id":40407916,"uuid":"269403820","full_name":"fullstorydev/solr-bench","owner":"fullstorydev","description":"Solr benchmarking and load testing harness","archived":false,"fork":false,"pushed_at":"2024-10-30T23:37:40.000Z","size":64104,"stargazers_count":17,"open_issues_count":13,"forks_count":10,"subscribers_count":24,"default_branch":"master","last_synced_at":"2024-10-31T00:24:37.391Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fullstorydev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-06-04T16:00:51.000Z","updated_at":"2024-10-30T23:37:44.000Z","dependencies_parsed_at":"2024-07-09T20:57:40.952Z","dependency_job_id":null,"html_url":"https://github.com/fullstorydev/solr-bench","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fullstorydev%2Fsolr-bench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fullstorydev%2Fsolr-bench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fullstorydev%2Fsolr-bench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fullstorydev%2Fsolr-bench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fullstorydev","download_url":"https://codeload.github.com/fullstorydev/solr-bench/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224354150,"owners_count":17297401,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T21:36:55.779Z","updated_at":"2024-11-12T21:36:56.402Z","avatar_url":"https://github.com/fullstorydev.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Solr Bench\n\nA comprehensive Solr performance benchmark and stress test framework.\n\nBenchmarking \u0026 stress test for standard operations (indexing, querying, collection operations, restarting nodes) or advanced tests (e.g. simulating GC pauses etc.) for a specified Solr build.\n\n## Benchmarking\n\n### Prerequisites\n\n#### GNU/Linux\n\n##### Local Mode\n\nUbuntu/Debian:\n\n    # Install JDK 11, make sure it is the default JDK. Following is a potential way to install the right JDK:\n    sudo apt install openjdk-11-jdk\n\n    sudo apt install wget unzip zip ant ivy lsof git netcat make maven jq\n    \nFedora/RHEL:\n\n    # Install JDK 11, make sure it is the default JDK. Following is a potential way to install the right JDK:\n    sudo yum install  java-11-openjdk-devel\n\n    sudo yum install wget unzip zip ant ivy lsof git nc make maven jq\n\n\n##### GCP Mode\nIf running on GCP, spin up a coordinator VM where this suite will run from. Make sure to use a service account for that VM that has permissions to create other VMs.\n\nThe VM should have the following:\n* Maven and other tools for building `apt install wget unzip zip ant ivy lsof git netcat make openjdk-11-jdk maven jq` (for Ubuntu/Debian) or `sudo yum install wget unzip zip ant ivy lsof git nc make java-11-openjdk-devel maven jq` (for Redhat/CentOS/Fedora)\n* Terraform. `wget https://releases.hashicorp.com/terraform/0.12.26/terraform_0.12.26_linux_amd64.zip; sudo unzip terraform_0.12.26_linux_amd64.zip -d /usr/local/bin`\n\n\n#### Mac OS\n\nTBD (PRs welcome!)\n\n### Running the suite\n\nIn the coordinator VM, check out this solr-bench repository.\n\n1. `mvn clean compile assembly:single`\n2. `./cleanup.sh \u0026\u0026 ./stress.sh -c \u003ccommit\u003e \u003cconfig-file\u003e`\n\nExample: `./cleanup.sh ./stress.sh -c dfde16a004206cc92e21cc5a6cad9030fbe13c20 suites/stress-facets-local.json`\n\n\n#### Available tests\n```\nNote: This is subject to change\n```\n\n1. `cluster-test.json` : Creates an 8 node cluster and create 1000 collections of various `numShards` and measure shutdown \u0026 restart performance\n2. `stress-facets-local.json` : Indexes 20 million documents from an ecommerce events dataset, issues 5k facet queries against it.\n3. `prs-vs-nonprs.json` : Creates 1k collections using non-PRS mode (default), then restarts 7 nodes, then cleans up, and repeats the same for PRS.\n\n### Results\n\n* Results are available after the benchmark in `./suites/results/\u003ctestname (without the .json suffix)\u003e/\u003ccommit-id\u003e/results.json` file.\n\n### Datasets\n\nTBD\n\n### Visualization\n\nTo select/process the test results of specific branch or branch comparisons:\n* `python3 graph-scripts/generate_graph_json.py -r suites/results/\u003ctest name\u003e -b main` (only select tests with commits that belongs to `main` branch)\n* `python3 graph-scripts/generate_graph_json.py -r suites/results/\u003ctest name\u003e -b main...my-branch` (compare tests between `main` and `my-branch`)\n\n```\nNote: The `\u003ctest name\u003e` currently is the test config file name without the `.json` file extension.\n\nNote: Many result directories can be specified at once, in case multiple tests have been performed. Example: `python3 graph-scripts/generate_graph_json.py -r suites/results/stress-facets-local -r suites/results/cluster-test -b branch_9x...branch_9_1`\n```\n\nThe script should generate `graph/graph-data.js`. Open `graph/graph.html`, it should show graphs grouped by branches and test tasks\n\n####  Dependencies on mac\nMac OS requires a few tools to run this script. Install the following:\n\n1. `brew install coreutils` \n\n### Prometheus Exporter\nCurrently, query and index benchmark metrics can also be exported via a Prometheus Exporter endpoint (default at port 11100 at /metrics).\n\nTo enable this, simply set `prometheus-export` node in the root level of the configuration, there are 2 optional fields under such node:\n- port : the port to run the http server which exposes the metrics, default 11100 (to avoid conflicts with node-exporter etc at 9100)\n- type-label : an extra label value with key \"type\" will be sent along the metrics, by default it has value \"all\"\n\nfor example:\n```\n{\n  ...\n  \"prometheus-export\": { \"port\": 1234, \"type-label\": \"facet\" }\n}\n```\nTo enable Prometheus with defaults, simply add `\"prometheus-export\": {}`\n\nLabel override can also defined as `prometheus-type-label` under `query-benchmark`\n\n\n### External mode with SSL\nFor testing when external mode (ie cluster.provisioning-method as \"external\"), which benchmarks against an external \n(existing) Solr cluster, we might want to set the solr-bench driver with extra JVM args for certs/keys location. \n\nSet at the root of the json config with `extra-jvm-args`, for example :\n`\"extra-jvm-args\": \"-Djavax.net.ssl.trustStore=\u003ctruststore path\u003e -Djavax.net.ssl.trustStorePassword=\u003csecret\u003e -Djavax.net.ssl.keyStore=\u003ckeystore path\u003e -Djavax.net.ssl.keyStorePassword=\u003csecret\u003e”`\n\n\n## Acknowledgement\nThis started as a project funded by Google Summer of Code (SOLR-10317), later supported by FullStory.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffullstorydev%2Fsolr-bench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffullstorydev%2Fsolr-bench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffullstorydev%2Fsolr-bench/lists"}