https://github.com/fullstorydev/solr-bench
Solr benchmarking and load testing harness
https://github.com/fullstorydev/solr-bench
Last synced: about 1 year ago
JSON representation
Solr benchmarking and load testing harness
- Host: GitHub
- URL: https://github.com/fullstorydev/solr-bench
- Owner: fullstorydev
- License: apache-2.0
- Created: 2020-06-04T16:00:51.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2024-10-30T23:37:40.000Z (over 1 year ago)
- Last Synced: 2024-10-31T00:24:37.391Z (over 1 year ago)
- Language: Java
- Size: 61.1 MB
- Stars: 17
- Watchers: 24
- Forks: 10
- Open Issues: 13
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Solr Bench
A comprehensive Solr performance benchmark and stress test framework.
Benchmarking & stress test for standard operations (indexing, querying, collection operations, restarting nodes) or advanced tests (e.g. simulating GC pauses etc.) for a specified Solr build.
## Benchmarking
### Prerequisites
#### GNU/Linux
##### Local Mode
Ubuntu/Debian:
# Install JDK 11, make sure it is the default JDK. Following is a potential way to install the right JDK:
sudo apt install openjdk-11-jdk
sudo apt install wget unzip zip ant ivy lsof git netcat make maven jq
Fedora/RHEL:
# Install JDK 11, make sure it is the default JDK. Following is a potential way to install the right JDK:
sudo yum install java-11-openjdk-devel
sudo yum install wget unzip zip ant ivy lsof git nc make maven jq
##### GCP Mode
If running on GCP, spin up a coordinator VM where this suite will run from. Make sure to use a service account for that VM that has permissions to create other VMs.
The VM should have the following:
* Maven and other tools for building `apt install wget unzip zip ant ivy lsof git netcat make openjdk-11-jdk maven jq` (for Ubuntu/Debian) or `sudo yum install wget unzip zip ant ivy lsof git nc make java-11-openjdk-devel maven jq` (for Redhat/CentOS/Fedora)
* Terraform. `wget https://releases.hashicorp.com/terraform/0.12.26/terraform_0.12.26_linux_amd64.zip; sudo unzip terraform_0.12.26_linux_amd64.zip -d /usr/local/bin`
#### Mac OS
TBD (PRs welcome!)
### Running the suite
In the coordinator VM, check out this solr-bench repository.
1. `mvn clean compile assembly:single`
2. `./cleanup.sh && ./stress.sh -c `
Example: `./cleanup.sh ./stress.sh -c dfde16a004206cc92e21cc5a6cad9030fbe13c20 suites/stress-facets-local.json`
#### Available tests
```
Note: This is subject to change
```
1. `cluster-test.json` : Creates an 8 node cluster and create 1000 collections of various `numShards` and measure shutdown & restart performance
2. `stress-facets-local.json` : Indexes 20 million documents from an ecommerce events dataset, issues 5k facet queries against it.
3. `prs-vs-nonprs.json` : Creates 1k collections using non-PRS mode (default), then restarts 7 nodes, then cleans up, and repeats the same for PRS.
### Results
* Results are available after the benchmark in `./suites/results///results.json` file.
### Datasets
TBD
### Visualization
To select/process the test results of specific branch or branch comparisons:
* `python3 graph-scripts/generate_graph_json.py -r suites/results/ -b main` (only select tests with commits that belongs to `main` branch)
* `python3 graph-scripts/generate_graph_json.py -r suites/results/ -b main...my-branch` (compare tests between `main` and `my-branch`)
```
Note: The `` currently is the test config file name without the `.json` file extension.
Note: Many result directories can be specified at once, in case multiple tests have been performed. Example: `python3 graph-scripts/generate_graph_json.py -r suites/results/stress-facets-local -r suites/results/cluster-test -b branch_9x...branch_9_1`
```
The script should generate `graph/graph-data.js`. Open `graph/graph.html`, it should show graphs grouped by branches and test tasks
#### Dependencies on mac
Mac OS requires a few tools to run this script. Install the following:
1. `brew install coreutils`
### Prometheus Exporter
Currently, query and index benchmark metrics can also be exported via a Prometheus Exporter endpoint (default at port 11100 at /metrics).
To enable this, simply set `prometheus-export` node in the root level of the configuration, there are 2 optional fields under such node:
- port : the port to run the http server which exposes the metrics, default 11100 (to avoid conflicts with node-exporter etc at 9100)
- type-label : an extra label value with key "type" will be sent along the metrics, by default it has value "all"
for example:
```
{
...
"prometheus-export": { "port": 1234, "type-label": "facet" }
}
```
To enable Prometheus with defaults, simply add `"prometheus-export": {}`
Label override can also defined as `prometheus-type-label` under `query-benchmark`
### External mode with SSL
For testing when external mode (ie cluster.provisioning-method as "external"), which benchmarks against an external
(existing) Solr cluster, we might want to set the solr-bench driver with extra JVM args for certs/keys location.
Set at the root of the json config with `extra-jvm-args`, for example :
`"extra-jvm-args": "-Djavax.net.ssl.trustStore= -Djavax.net.ssl.trustStorePassword= -Djavax.net.ssl.keyStore= -Djavax.net.ssl.keyStorePassword=”`
## Acknowledgement
This started as a project funded by Google Summer of Code (SOLR-10317), later supported by FullStory.