{"id":13583818,"url":"https://github.com/cloud-bulldozer/benchmark-operator","last_synced_at":"2025-12-14T23:04:13.582Z","repository":{"id":37963490,"uuid":"172605422","full_name":"cloud-bulldozer/benchmark-operator","owner":"cloud-bulldozer","description":"The Chuck Norris of cloud benchmarks","archived":false,"fork":false,"pushed_at":"2024-06-20T09:46:30.000Z","size":9890,"stargazers_count":282,"open_issues_count":6,"forks_count":127,"subscribers_count":21,"default_branch":"master","last_synced_at":"2024-08-02T15:53:55.080Z","etag":null,"topics":["kubernetes","kubernetes-operator","openshift","openshift-operator","performance-testing"],"latest_commit_sha":null,"homepage":"","language":"Jinja","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloud-bulldozer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-26T00:03:27.000Z","updated_at":"2024-07-04T08:49:10.000Z","dependencies_parsed_at":"2023-02-11T01:35:17.107Z","dependency_job_id":"e5e707f0-5439-4ec3-89e9-159298bb573f","html_url":"https://github.com/cloud-bulldozer/benchmark-operator","commit_stats":null,"previous_names":["cloud-bulldozer/ripsaw"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/cloud-bulldozer/benchmark-operator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud-bulldozer%2Fbenchmark-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud-bulldozer%2Fbenchmark-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud-bulldozer%2Fbenchmark-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud-bulldozer%2Fbenchmark-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloud-bulldozer","download_url":"https://codeload.github.com/cloud-bulldozer/benchmark-operator/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud-bulldozer%2Fbenchmark-operator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27738659,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-14T02:00:11.348Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kubernetes","kubernetes-operator","openshift","openshift-operator","performance-testing"],"created_at":"2024-08-01T15:03:49.745Z","updated_at":"2025-12-14T23:04:13.548Z","avatar_url":"https://github.com/cloud-bulldozer.png","language":"Jinja","funding_links":[],"categories":["Jinja"],"sub_categories":[],"readme":"# Benchmark Operator\n\nThe intent of this Operator is to deploy common workloads to establish\na performance baseline of Kubernetes cluster on your provider.\n\n\n## Installation (Default)\nThe easiest way to install the operator is through the operator-sdk methods provided in the `Makefile`.\n\n```bash\ngit clone https://github.com/cloud-bulldozer/benchmark-operator\ncd benchmark-operator\nmake deploy\n```\n\nWatch how to do this on Youtube:\n\n[![Benchmark Operator Quickstart](http://img.youtube.com/vi/eQxKl_uve84/0.jpg)](http://www.youtube.com/watch?v=eQxKl_uve84 \"Benchmark Operator Quickstart\")\n\nIf you wish to build a version of the operator from your local copy of the repo, you can run\n\n```bash\ngit clone https://github.com/cloud-bulldozer/benchmark-operator\ncd benchmark-operator\nmake image-build image-push deploy IMG=$YOUR_IMAGE\n```\n\n\u003e Note: building the image requires podman\n\n## Installation (Helm)\n\nInstalling the benchmark-operator via Helm can be done with the following commands. This requires\nyour machine to have Helm installed. [Install Helm](https://helm.sh/docs/intro/install/)\n\n\u003e Note: If running on openshift you'll need to run this command before installing the chart. `oc adm policy -n benchmark-operator add-scc-to-user privileged -z benchmark-operator`\n\n\n\n```bash\ngit clone https://github.com/cloud-bulldozer/benchmark-operator\ncd benchmark-operator/charts/benchmark-operator\nkubectl create namespace benchmark-operator\noc adm policy -n benchmark-operator add-scc-to-user privileged -z benchmark-operator # Openshift Only\nhelm install benchmark-operator . -n benchmark-operator --create-namespace\n```\n\nTo delete this release, you can do so with the following command:\n\n```bash\nhelm uninstall benchmark-operator -n benchmark-operator\n```\n\n\n\n## Workloads status\n\n| Workload                       | Use                    | ElasticSearch indexing  | Reconciliation usage       | VM support (kubevirt) | Kata Containers | CI Tested |\n| ------------------------------ | ---------------------- | ------------------ | -------------------------- | --------------------- | --------------- | ------------ |\n| [UPerf](docs/uperf.md)         | Network Performance    | Yes                |  Used, default : 3second  | Working                | Working         | Yes |\n| [Iperf3](docs/iperf3.md)       | Network Performance     | No                 |  Used, default : 3second  | Not Supported          | Preview         | Yes |\n| [fio](docs/fio_distributed.md) | Storage IO             | Yes                |  Used, default : 3second  | Working                | Working         | Yes |\n| [Sysbench](docs/sysbench.md)   | System Performance     | No                 |  Used, default : 3second  | Not Supported          | Preview         | Yes |\n| [YCSB](docs/ycsb.md)           | Database Performance   | Yes            |  Used, default : 3second  | Not Supported          | Preview         | Yes |\n| [Byowl](docs/byowl.md)         | User defined workload  | Yes            |  Used, default : 3second  | Not Supported          | Preview         | Yes |\n| [Pgbench](docs/pgbench.md)     | Postgres Performance   | Yes            |  Used, default : 3second  | Not Supported          | Preview         | Yes |\n| [Smallfile](docs/smallfile.md) | Storage IO Performance | Yes            |  Used, default : 3second  | Not Supported          | Preview         | Yes |\n| [fs-drift](docs/fs-drift.md)   | Storage IO Longevity   | Yes            |  Not used                 | Not Supported          | Preview         | Yes |\n| [hammerdb](docs/hammerdb.md)   | Database Performance   | Yes            |  Used, default : 3second  |  Working                | Preview         | Yes |\n| [Service Mesh](docs/servicemesh.md) | Microservices     | No            |  Used, default : 3second   | Not Supported         | Preview         | No |\n| [Vegeta](docs/vegeta.md)       | HTTP Performance       | Yes            |  Used, default : 3second  | Not Supported          | Preview         | Yes |\n| [Scale Openshift](docs/scale_openshift.md) | Scale Openshift Cluster       | Yes            |  Used, default : 3second  | Not Supported         | Preview        | Yes |\n| [stressng](docs/stressng.md)   | Stress system resources | Yes            |  Used, default: 3second  | Working               | Preview        | Yes |\n| [kube-burner](docs/kube-burner.md)  | k8s Performance   | Yes            |  Used, default : 3second  | Not Supported          | Preview         | Yes |\n| [cyclictest](docs/cyclictest.md)  | Real-Time Performance   | Yes       |  Used, default : 3second  | Not Supported          | Preview         | No |\n| [oslat](docs/oslat.md)         | Real-Time Latency      | Yes           |  Used, default : 3second   | Not Supported          | Preview         | No |\n| [testpmd](docs/testpmd.md)         | TestPMD DPDK App      | No           |  Used   | Not Supported          | Preview         | No |\n| [Flent](docs/flent.md)         | Network Performance    | Yes           |  Used, default : 3second  | Not Supported          | Not Supported   | Yes |\n| [Log-Generator](docs/log_generator.md)         | Log Throughput to Backend    | Yes           |  Used, default : 3second  | Not Supported          | Yes  | Yes |\n| [Image-Pull](docs/image_pull.md)         | Time to Pull Image from Container Repo    | Yes           |  Used, default : 3second  | Not Supported          | Yes  | Yes |\n\n### Reconciliation\n\nPreviously the Benchmark Operator didn't properly take advantage of the reconciliation period. Going forward\nwe will make every attempt to utilize the reconciliation period.\n\nWhy did we decide to switch to this? Our operator would implement long running tasks, due to the nature of benchmarks.\nHowever, long running tasks blocks the Operator, causing us to delete the Operator and re-create the operator to\nun-block it. The benchmarks mentioned above that state `Used` for Reconciliation, no longer have this issue.\n\n# E2E tests\nBenchmark-operator includes a series of end 2 end tests that can be triggered in local. More info in the [documentation.](docs/e2e-ci.md#running-in-local)\n\n## Optional workload images\nOptional locations for workload images can now be added easily without the need to rebuild the operator.\nTo do so in the workload args section of the CR add image: [location]\n\nNOTE: This is not a required argument. If omitted it will default to the currently verified workload image.\nAdditionally, this is *NOT* enabled for YCSB\n\nFor Example:\n\n```\napiVersion: ripsaw.cloudbulldozer.io/v1alpha1\nkind: Benchmark\nmetadata:\n  name: example-benchmark\n  namespace: benchmark-operator\nspec:\n  elasticsearch:\n    url: \"http://my-es.foo.bar:80\"\n  metadata_collection: true\n  cleanup: false\n  workload:\n    name: \"foo\"\n    args:\n      image: my.location/foo:latest\n```\n\n## Optional debug out for benchmark-wrapper workloads\nWorkloads that are triggered through [benchmark-wrapper](https://github.com/cloud-bulldozer/benchmark-wrapper)\ncan optionally pass the debug flag through the workload CR.\n\nNOTE: This is not a required argument. If omitted it will default to the default logging level of\nthe benchmark-wrapper. \n\nFor Example:\n\n```\napiVersion: ripsaw.cloudbulldozer.io/v1alpha1\nkind: Benchmark\nmetadata:\n  name: example-benchmark\n  namespace: benchmark-operator\nspec:\n  elasticsearch:\n    url: \"http://my-es.foo.bar:80\"\n  metadata_collection: true\n  cleanup: false\n  workload:\n    name: snafu_workload\n    args:\n      debug: true\n```\n\n## User Provided UUID\nAll benchmarks in the benchmark-operator utilize a UUID for tracking and indexing purposes. This UUID is,\nby default, generated when the workload is first started. However, if desired, a user provided UUID can\nbe added to the workload cr.\n\n*NOTE: The provided UUID must be in format XXXXX-XXXXX-XXXXX*\n\nFor Example:\n```\napiVersion: ripsaw.cloudbulldozer.io/v1alpha1\nkind: Benchmark\nmetadata:\n  name: example-benchmark\n  namespace: benchmark-operator\nspec:\n  uuid: 6060004a-7515-424e-93bb-c49844600dde\n  elasticsearch:\n    url: \"http://my-es.foo.bar:80\"\n  metadata_collection: true\n  cleanup: false\n  workload:\n    name: \"foo\"\n    args:\n      image: my.location/foo:latest\n```\n\n## Contributing\n[Contributing](CONTRIBUTING.md)\n\n## Metadata Collection\n[Metadata Collection](docs/metadata.md)\n\n## Indexing to Elasticsearch\n[Indexing to Elasticsearch](docs/elastic.md)\n\n## Capturing Prometheus Data\n[Capturing Prometheus Data](docs/prometheus.md)\n\n## Cache dropping\n[Cache dropping](docs/cache_dropping.md)\n\n## Community\nKey Members(slack_usernames):  sejug, mohit, dry923, rsevilla or rook\n* [**#sig-scalability on Kubernetes Slack**](https://kubernetes.slack.com)\n* [**#forum-perfscale on CoreOS Slack**](https://coreos.slack.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloud-bulldozer%2Fbenchmark-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloud-bulldozer%2Fbenchmark-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloud-bulldozer%2Fbenchmark-operator/lists"}