{"id":50385035,"url":"https://github.com/stackhpc/kube-perftest","last_synced_at":"2026-05-30T14:30:54.513Z","repository":{"id":43786352,"uuid":"411231416","full_name":"stackhpc/kube-perftest","owner":"stackhpc","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-01T15:21:57.000Z","size":767,"stargazers_count":16,"open_issues_count":8,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-05-01T17:17:58.832Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stackhpc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-09-28T10:13:53.000Z","updated_at":"2026-05-01T15:22:02.000Z","dependencies_parsed_at":"2023-02-16T08:31:20.178Z","dependency_job_id":"9e8499f1-23b3-4e65-b959-0346fe21a842","html_url":"https://github.com/stackhpc/kube-perftest","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/stackhpc/kube-perftest","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackhpc%2Fkube-perftest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackhpc%2Fkube-perftest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackhpc%2Fkube-perftest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackhpc%2Fkube-perftest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stackhpc","download_url":"https://codeload.github.com/stackhpc/kube-perftest/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackhpc%2Fkube-perftest/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33696681,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-30T02:00:06.278Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-30T14:30:53.908Z","updated_at":"2026-05-30T14:30:54.508Z","avatar_url":"https://github.com/stackhpc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kube-perftest  \u003c!-- omit in toc --\u003e\n\n`kube-perftest` is a framework for building and running performance benchmarks for\n[Kubernetes](https://kubernetes.io/) clusters.\n\n- [Installation](#installation)\n- [Network selection](#network-selection)\n- [Benchmark set](#benchmark-set)\n- [Benchmarks](#benchmarks)\n  - [iperf](#iperf)\n  - [MPI PingPong](#mpi-pingpong)\n  - [OpenFOAM](#openfoam)\n  - [RDMA Bandwidth](#rdma-bandwidth)\n  - [RDMA Latency](#rdma-latency)\n  - [fio](#fio)\n  - [PyTorch](#PyTorch)\n- [Operator development](#operator-development)\n\n## Installation\n\nRequires a volcano.sh installation on the cluster, can be installed using\n\n```sh\nkubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/master/installer/volcano-development.yaml\n```\n\nThe `kube-perftest` operator can be installed using [Helm](https://helm.sh):\n\n```sh\nhelm repo add kube-perftest https://stackhpc.github.io/kube-perftest\n# Use the most recent published chart for the main branch\nhelm upgrade \\\n  kube-perftest-operator \\\n  kube-perftest/kube-perftest-operator \\\n  -i \\\n  --version \"\u003e=0.1.0-dev.0.main.0,\u003c0.1.0-dev.0.main.99999999999\"\n```\n\nFor most use cases, no customisations to the Helm values will be necessary.\n\n## Network selection\n\nAll the benchmarks are capable of running using the Kubernetes pod network or the host network\n(using `hostNetwork: true` on the benchmark pods).\n\nBenchmarks are also able to run on accelerated networks where available, using\n[Multus](https://github.com/k8snetworkplumbingwg/multus-cni) for multiple CNIs and\n[device plugins](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/)\nto request network resources.\n\nThis allows benchmarks to levarage technologies such as\n[SR-IOV](https://en.wikipedia.org/wiki/Single-root_input/output_virtualization)\n(via the [SR-IOV network device plugin](https://github.com/k8snetworkplumbingwg/sriov-network-device-plugin)),\n[macvlan](https://backreference.org/2014/03/20/some-notes-on-macvlanmacvtap/) (via the\n[macvlan CNI plugin](https://www.cni.dev/plugins/current/main/macvlan/)) and\n[RDMA](https://en.wikipedia.org/wiki/Remote_direct_memory_access)\n(e.g. via the [RDMA shared device plugin](https://github.com/Mellanox/k8s-rdma-shared-dev-plugin)).\n\nThe networking is configured using the following properties of the benchmark `spec`:\n\n```yaml\nspec:\n  # Indicates whether to use host networking or not\n  # If true, networkName is not used\n  hostNetwork: false\n  # The name of a Multus network to use\n  # Only used if hostNetwork is false\n  # If not given, the Kubernetes pod network is used\n  networkName: namespace/netname\n  # The resources for benchmark pods\n  resources:\n    limits:\n      # E.g. requesting a share of an RDMA device\n      rdma/hca_shared_devices_a: 1\n  # The MTU to set on the interface *inside* the container\n  # If not given, the default MTU is used\n  mtu: 9000\n```\n\n## Benchmark set\n\nThe `kube-perftest` operator provides a `BenchmarkSet` resource that can be used to run\nthe same benchmark over a sweep of parameters:\n\n```yaml\napiVersion: perftest.stackhpc.com/v1alpha1\nkind: BenchmarkSet\nmetadata:\n  name: iperf\nspec:\n  # The template for the fixed parts of the benchmark\n  template:\n    apiVersion: perftest.stackhpc.com/v1alpha1\n    kind: IPerf\n    spec:\n      duration: 30\n  # The number of repetitions to run for each permutation\n  # Defaults to 1 if not given\n  repetitions: 1\n  # Defines the permutations for the set\n  # Each permutation is merged into the spec of the template\n  # If not given, a single permutation consisting of the given template is used\n  permutations:\n    # Permutations are calculated as a cross-product of the specified names and values\n    product:\n      hostNetwork: [true, false]\n      streams: [1, 2, 4, 8, 16, 32, 64]\n    # A list of explicit permutations to include\n    explicit:\n      - hostNetwork: true\n        streams: 128\n```\n\n## Benchmarks\n\nCurrently, the following benchmarks are supported:\n\n### iperf\n\nRuns the [iperf](https://en.wikipedia.org/wiki/Iperf) network performance tool to measure bandwidth\nfor a transfer between two pods.\n\n```yaml\napiVersion: perftest.stackhpc.com/v1alpha1\nkind: IPerf\nmetadata:\n  name: iperf\nspec:\n  # The number of parallel streams to use\n  streams: 8\n  # The duration of the test\n  duration: 30\n```\n\n### MPI PingPong\n\nRuns the\n[Intel MPI Benchmarks (IMB) MPI1 PingPong](https://www.intel.com/content/www/us/en/develop/documentation/imb-user-guide/top/mpi-1-benchmarks/single-transfer-benchmarks/pingpong-pingpongspecificsource-pingponganysource.html)\nbenchmark to measure the average round-trip time and bandwidth for MPI messages of different sizes\nbetween two pods.\n\nUses [Open MPI](https://www.open-mpi.org/) initialised over SSH. The data plane can use TCP\nor, hardware and network permitting, [RDMA](https://en.wikipedia.org/wiki/Remote_direct_memory_access)\nvia [UCX](https://openucx.org/).\n\n```yaml\napiVersion: perftest.stackhpc.com/v1alpha1\nkind: MPIPingPong\nmetadata:\n  name: mpi-pingpong\nspec:\n  # The MPI transport to use - one of TCP, RDMA\n  transport: TCP\n  # Controls the maximum message length\n  # Selected lengths, in bytes, will be 0, 1, 2, 4, 8, 16, ..., 2^maxlog\n  # Defaults to 22 if not given, meaning the maximum message size will be 4MB\n  maxlog: 22\n```\n\n### OpenFOAM\n\n[OpenFOAM](https://www.openfoam.com/) is a toolbox for solving problems in\n[computational fluid dynamics (CFD)](https://en.wikipedia.org/wiki/Computational_fluid_dynamics).\nIt is included here as an example of a \"real world\" workload.\n\nThis benchmark runs the\n[3-D Lid Driven cavity flow benchmark](https://develop.openfoam.com/committees/hpc#3-d-lid-driven-cavity-flow)\nfrom the OpenFOAM benchmark suite.\n\nUses [Open MPI](https://www.open-mpi.org/) initialised over SSH. The data plane can use TCP\nor, hardware and network permitting, [RDMA](https://en.wikipedia.org/wiki/Remote_direct_memory_access)\nvia [UCX](https://openucx.org/).\n\n```yaml\napiVersion: perftest.stackhpc.com/v1alpha1\nkind: OpenFOAM\nmetadata:\n  name: openfoam\nspec:\n  # The MPI transport to use - one of TCP, RDMA\n  transport: TCP\n  # The problem size to use - one of S, M, XL, XXL\n  problemSize: S\n  # The number of MPI processes to use\n  numProcs: 16\n  # The number of MPI pods to launch\n  numNodes: 8\n```\n\n### RDMA Bandwidth\n\nRuns the RDMA bandwidth benchmarks (i.e. `ib_{read,write}_bw`) from the\n[perftest collection](https://github.com/linux-rdma/perftest).\n\nThis benchmark requires an RDMA-capable network to be specified.\n\n```yaml\napiVersion: perftest.stackhpc.com/v1alpha1\nkind: RDMABandwidth\nmetadata:\n  name: rdma-bandwidth\nspec:\n  # The mode for the test - read or write\n  mode: read\n  # The number of iterations to do at each message size\n  # Defaults to 1000 if not given\n  iterations: 1000\n  # The number of queue pairs to use\n  # Defaults to 1 if not given\n  # A higher number of queue pairs can help to spread traffic,\n  # e.g. over NICs in a bond when using RoCE-LAG\n  qps: 1\n  # Extra arguments to be added to the command\n  extraArgs:\n    - --tclass=96\n```\n\n### RDMA Latency\n\nRuns the RDMA latency benchmarks (i.e. `ib_{read,write}_lat`) from the\n[perftest collection](https://github.com/linux-rdma/perftest).\n\nThis benchmark requires an RDMA-capable network to be specified.\n\n```yaml\napiVersion: perftest.stackhpc.com/v1alpha1\nkind: RDMALatency\nmetadata:\n  name: rdma-latency\nspec:\n  # The mode for the test - read or write\n  mode: read\n  # The number of iterations to do at each message size\n  # Defaults to 1000 if not given\n  iterations: 1000\n  # Extra arguments to be added to the command\n  extraArgs:\n    - --tclass=96\n```\n\n### fio\n\nRuns filesystem performance benchmarking using [fio](https://fio.readthedocs.io) to\ndetermine filesystem performance characteristics. All available `spec` options are\ngiven below. Fio configration options match broadly with those defined in the fio\ndocumentation.\n\nSetting `.spec.volumeClaimTemplate` allows the provision of stable storage using \n`PersistentVolumes` provisioned by a [`PersistentVolume`](https://kubernetes.io/docs/concepts/storage/persistent-volumes/)\nProvisioner.\n\nWhen `spec.volumeClaimTemplate.accessModes` contains `ReadWriteMany`, this benchmark\nwill create a single `PersistentVolume` per `BenchmarkSet` iteration, and attach all\nworker pods participting in the set (equal to `spec.numWorkers`) to the same volume.\nOtherwise, a `PersistentVolume` per-pod is created and attached to each worker pod\nparticipating in the benchmark.\n\n```yaml\napiVersion: perftest.stackhpc.com/v1alpha1\nkind: Fio\nmetadata:\n  name: fio-filesystem\nspec:\n  # fio benchmark configuration options\n  direct: 1\n  iodepth: 8\n  ioengine: libaio\n  nrfiles: 1\n  numJobs: 1\n  bs: 1M\n  rw: read\n  percentageRandom: 100\n  runtime: 10s\n  rwmixread: 50\n  size: 1G\n\n  # kube-perftest benchmark configuration\n  # options\n  numWorkers: 1\n  thread: false\n  hostNetwork: false\n\n  # PersistentVolume configuration options\n  volumeClaimTemplate:\n    accessModes:\n      - ReadWriteOnce\n    storageClassName: csi-cinder\n    resources:\n      requests:\n        storage: 5Gi\n```\n\n### PyTorch\n\nRuns machine learning model training and inference micro-benchmarks from the official \nPyTorch [benchmarks repo](https://github.com/pytorch/benchmark/) to compare performance\nof CPU and GPU devices on synthetic input data. Running benchmarks on CUDA-capable\ndevices requires the [Nvidia GPU Operator](https://github.com/NVIDIA/gpu-operator) \nto be pre-installed on the target Kubernetes cluster.\n\nThe pre-built container image currently includes the `alexnet`, `resnet50` and \n`llama` (inference only) models - additional models from the \n[upstream repo list](https://github.com/pytorch/benchmark/tree/main/torchbenchmark/models)\nmay be added as needed in the future. (Adding a new model simply requires adding it to the list\nin `images/pytorch-benchmark/Dockerfile` and updating the `PyTorchModel` enum in `pytorch.py`.)\n\n```yaml\napiVersion: perftest.stackhpc.com/v1alpha1\nkind: PyTorch\nmetadata:\n  name: pytorch-test-gpu\nspec:\n  # The device to run the benchmark on ('cpu' or 'cuda')\n  device: cuda\n  # Name of model to benchmark\n  model: alexnet\n  # Either 'train' or 'eval'\n  # (not all models support both)\n  benchmarkType: eval\n  # Batch size for generated input data\n  inputBatchSize: 32\n  resources:\n    limits:\n      nvidia.com/gpu: 2\n```\n\n\n## Operator development\n\n```\n# Install dependencies in a virtual environment\npython3 -m venv venv\nsource venv/bin/activate\npip install -U pip\npip install -r python/requirements.txt\npip install -e python\n\n# Set the default image tag\nexport KUBE_PERFTEST__DEFAULT_IMAGE_TAG=\u003cdev branch name\u003e\n\n# Set the default image pull policy\nexport KUBE_PERFTEST__DEFAULT_IMAGE_PULL_POLICY=Always\n\n# Run the operator\nkopf run -m perftest.operator -A\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstackhpc%2Fkube-perftest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstackhpc%2Fkube-perftest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstackhpc%2Fkube-perftest/lists"}