{"id":13586412,"url":"https://github.com/distributed-system-analysis/pbench","last_synced_at":"2026-04-04T15:43:47.632Z","repository":{"id":34079177,"uuid":"37891737","full_name":"distributed-system-analysis/pbench","owner":"distributed-system-analysis","description":"A benchmarking and performance analysis framework","archived":false,"fork":false,"pushed_at":"2024-09-25T19:47:18.000Z","size":402050,"stargazers_count":190,"open_issues_count":405,"forks_count":108,"subscribers_count":27,"default_branch":"main","last_synced_at":"2025-04-07T15:43:49.166Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://distributed-system-analysis.github.io/pbench/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/distributed-system-analysis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-06-23T02:08:30.000Z","updated_at":"2025-02-18T12:28:15.000Z","dependencies_parsed_at":"2023-10-21T10:55:35.789Z","dependency_job_id":"b6d5040c-97e0-486e-89f2-5acc5307a9b9","html_url":"https://github.com/distributed-system-analysis/pbench","commit_stats":null,"previous_names":[],"tags_count":74,"template":false,"template_full_name":null,"purl":"pkg:github/distributed-system-analysis/pbench","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/distributed-system-analysis%2Fpbench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/distributed-system-analysis%2Fpbench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/distributed-system-analysis%2Fpbench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/distributed-system-analysis%2Fpbench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/distributed-system-analysis","download_url":"https://codeload.github.com/distributed-system-analysis/pbench/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/distributed-system-analysis%2Fpbench/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31403960,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T10:20:44.708Z","status":"ssl_error","status_checked_at":"2026-04-04T10:20:06.846Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:05:33.373Z","updated_at":"2026-04-04T15:43:47.610Z","avatar_url":"https://github.com/distributed-system-analysis.png","language":"Python","funding_links":[],"categories":["Python","Shell"],"sub_categories":[],"readme":"# Pbench\nA Benchmarking and Performance Analysis Framework\n\nThe code base includes three sub-systems. The first is the collection agent,\n[Pbench Agent](docs/Agent/agent.md), responsible for collecting configuration\ndata for test systems, managing the collection of performance tool data from\nthose systems (`sar`, `vmstat`, `perf`, etc.), and executing and postprocessing\nstandardized or arbitrary benchmarked workloads (`uperf`, `fio`, `linpack`, as\nwell as real system activity).\n\nThe second sub-system is the [Pbench Server](docs/Server/server.md), which is\nresponsible for archiving result tar balls and providing a secure\n[RESTful API](docs/Server/API/README.md) to client applications, such as the\nPbench Dashboard. The API supports curation of results data, the ability to\nannotate results with arbitrary metadata, and to explore the results and\ncollected data.\n\nThe third sub-system is the [Pbench Dashboard](docs/Dashboard/user_guide.md), which\nprovides a web-based GUI for the Pbench Server allowing users to list and view\npublic results. After logging in, users can view their own results, publish\nresults for others to view, and delete results which are no longer of use.\nOn the _User Profile_ page, a logged-in user can generate API keys for use with\nthe Pbench Server API or with the Agent `pbench-results-move` command. The\nPbench Dashboard also serves as a platform for exploring and visualizing result\ndata.\n\n## How is it installed?\nInstructions for installing `pbench-agent`, can be found\nin the Pbench Agent [Getting Started Guide](\nhttps://distributed-system-analysis.github.io/pbench/gh-pages/start.html).\n\nFor Fedora, CentOS, and RHEL users, we have made available [COPR RPM\nbuilds](https://copr.fedorainfracloud.org/coprs/ndokos/pbench/) for the\n`pbench-agent` and some benchmark and tool packages.\n\nYou might want to consider browsing through the [rest of the documentation](\nhttps://distributed-system-analysis.github.io/pbench/gh-pages/doc.html).\n\nYou can also use `podman` or `docker` to pull Pbench Agent containers from\n[Quay.io](https://quay.io/pbench/).\n\n## How do I use pbench?\nRefer to the [Pbench Agent Getting Started Guide](\nhttps://distributed-system-analysis.github.io/pbench/gh-pages/start.html).\n\nTL;DR? See \"[TL;DR - How to set up the `pbench-agent` and run a benchmark\n](https://distributed-system-analysis.github.io/pbench/gh-pages/doc.html#how)\" in the\nmain documentation for a super quick set of introductory steps.\n\n## Where is the source kept?\nThe latest source code is at\nhttps://github.com/distributed-system-analysis/pbench.\n\n## Is there a mailing list for discussions?\n\nYes, we use [Google Groups](https://groups.google.com/forum/#!forum/pbench)\n\n## How do I report an issue?\n\nPlease use GitHub's [issues](\nhttps://github.com/distributed-system-analysis/pbench/issues/new/choose).\n\n## Is there a place to track current and future work items?\n\nYes, we are using GitHub [Projects](\nhttps://github.com/distributed-system-analysis/pbench/projects).\nPlease find projects covering the [Agent](\nhttps://github.com/distributed-system-analysis/pbench/projects/2),\n[Server](https://github.com/distributed-system-analysis/pbench/projects/3),\n[Dashboard](https://github.com/distributed-system-analysis/pbench/projects/1),\nand a project that is named the same as the current [milestone](\nhttps://github.com/distributed-system-analysis/pbench/milestones).\n\n## How can I contribute?\n\nBelow are some simple steps for setting up a development environment for\nworking with the Pbench code base.  For more detailed instructions on the\nworkflow and process of contributing code to Pbench, refer to the [Guidelines\nfor Contributing](docs/Developers/contributing.md).\n\n### Getting the Code\n\n```\n$ git clone https://github.com/distributed-system-analysis/pbench\n$ cd pbench\n```\n\n### Running the Unit Tests\n\nInstall `tox` properly in your environment (Fedora/CentOS/RHEL):\n\n```\n$ sudo dnf install -y perl-JSON python3-pip python3-tox\n```\n\nOnce tox is installed you can run the unit tests against different versions of\npython using the python environment short-hands:\n\n  * `tox -e py36`    -- run all tests in a Python 3.6 environment (our default)\n  * `tox -e py39`    -- run all tests in a Python 3.9 environment\n  * `tox -e py310`   -- run all tests in a Python 3.10 environment\n  * `tox -e pypy3`   -- run all tests in a PyPy 3 environment\n  * `tox -e pypy3.8` -- run all tests in a PyPy 3.8 environment\n\nSee https://tox.wiki/en/latest/example/basic.html#a-simple-tox-ini-default-environments.\n\nYou can provide arguments to the `tox` invocation to request sub-sets of the\navailable tests be run.\n\nFor example, if you want to just run the agent or server tests, you'd invoke\n`tox` as follows:\n\n  * `tox -- agent`   -- runs only the agent tests\n  * `tox -- server`  -- runs only the server tests\n\nEach of the \"agent\" and \"server\" tests can be further subsetted as follows:\n\n  * agent\n    * python          -- runs the python tests (via `pytest`)\n    * legacy          -- runs all the legacy tests\n    * datalog         -- runs only the legacy tool data-log tests,\n                         `agent/tool-scripts/datalog/unittests`\n    * postprocess     -- runs only the legacy tool/bench-scripts post-processing\n                         tests, `agent/tool-scripts/postprocess/unittests`\n    * tool-scripts    -- runs only the legacy tool-scripts tests,\n                         `agent/tool-scripts/unittests`\n    * util-scripts    -- runs only the legacy util-scripts tests,\n                         `agent/util-scripts/unittests`\n    * bench-scripts   -- runs only the legacy bench-scripts tests,\n                         `agent/bench-scripts/unittests`\n\n  * server\n    * python          -- runs the python tests (via python)\n\nFor example:\n\n  * `tox -- agent legacy`   -- run agent legacy tests\n  * `tox -- server python`  -- run server python tests (via `pytest`)\n\nFor any of the test sub-sets on either the agent or server sides of the tree,\none can pass additional arguments to the specific sub-system test runner.  This\nallows one to request a specific test, or set of tests, or command line\nparameters to modify the test behavior:\n\n  * `tox -- agent bench-scripts test-CL`    -- run bench-scripts' test-CL\n  * `tox -- server python -v`               -- run server python tests verbosely\n\nFor the `agent/bench-scripts` tests, one can run entire sub-sets of tests using\na sub-directory name found in `agent/bench-scripts/tests`. For example:\n\n  * `tox -- agent bench-scripts pbench-fio`\n  * `tox -- agent bench-scripts pbench-uperf pbench-linpack`\n\nThe first runs all the `pbench-fio` tests, while the second runs all the\n`pbench-uperf` and `pbench-linpack` tests.\n\nYou can run the `build.sh` script to execute the linters, to run the unit tests\nfor the Agent, Server, and Dashboard code, and to build installations for the\nAgent, Server, and Dashboard.\n\nFinally, see the `jenkins/Pipeline.gy` file for how the unit tests are run in\nour CI jobs.\n\n### Python formatting\n\nThis project uses the [`flake8`](http://flake8.pycqa.org/en/latest) method of code\nstyle enforcement, linting, and checking.\n\nAll python code contributed to pbench must match the style requirements. These\nrequirements are enforced by the [pre-commit](https://pre-commit.com) hook.  In\naddition to `flake8`, pbench uses the [`black`](https://github.com/psf/black)\nPython code formatter and the [`isort`](https://github.com/pycqa/isort) Python\nimport sorter.\n\n### Use pre-commit to set automatic commit requirements\n\nThis project makes use of [pre-commit](https://pre-commit.com/) to do automatic\nlint and style checking on every commit containing Python files.\n\nTo install the pre-commit hook, run the executable from your Python 3 framework\nwhile in your current pbench git checkout:\n\n```\n$ cd ~/pbench\n$ pip3 install pre-commit\n$ pre-commit install --install-hooks\n```\n\nOnce installed, all commits will run the test hooks. If your changes fail any of\nthe tests, the commit will be rejected.\n\n## Pbench Release Tag Scheme (GitHub)\nWe employ a simple major, minor, release, build (optional) scheme for tagging\nstarting with the `v0.70.0` release (`v\u003cMajor\u003e.\u003cMinor\u003e.\u003cRelease\u003e[-\u003cBuild\u003e]`).\nPrior to the v0.70.0 release, the scheme used was mostly `v\u003cMajor\u003e.\u003cMinor\u003e`,\nwhere we only had minor releases (`Major = 0`).\n\n### Container Image Tags\nThis same GitHub \"tag\" scheme is used with tags applied to container images\nwe build, with the following exceptions for tag names:\n\n  * `latest` - always points to the latest released container image pushed to a\n    repository\n\n  * `v\u003cMajor\u003e-latest` - always points to the \"latest\" `Major` released\n    image\n\n  * `v\u003cMajor\u003e.\u003cMinor\u003e-latest` - always points to the \"latest\" release\n    for `Major`.`Minor` released images\n\n  * `\u003cSHA1 git hash\u003e` (9 characters) - commit hash of the checked out code\n\n### References to Container Image Repositories\nThe operation of our functional tests, the Pbench Server \"in-a-can\" used in\nthe functional tests, and other verification and testing environments use\ncontainer images from remote image registries.  The CI jobs\nobtain references to those repositories using Jenkins credentials.  When\nrunning those same jobs locally, you can provide the registry via\n`${HOME}/.config/pbench/ci_registry.name`.\n\nIf this file is not provided, local execution will report an error.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdistributed-system-analysis%2Fpbench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdistributed-system-analysis%2Fpbench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdistributed-system-analysis%2Fpbench/lists"}