{"id":18794431,"url":"https://github.com/chainstacklabs/chainbench","last_synced_at":"2025-04-07T16:17:52.172Z","repository":{"id":154900228,"uuid":"629643962","full_name":"chainstacklabs/chainbench","owner":"chainstacklabs","description":"Flexible blockchain infrastructure benchmarking and load testing tool","archived":false,"fork":false,"pushed_at":"2025-03-06T05:04:08.000Z","size":773,"stargazers_count":42,"open_issues_count":4,"forks_count":12,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-03-31T14:11:13.390Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chainstacklabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-18T18:23:58.000Z","updated_at":"2025-03-29T12:36:16.000Z","dependencies_parsed_at":"2024-01-19T09:35:25.069Z","dependency_job_id":"340352ba-9c6f-4f7d-8b42-a9502f0f1608","html_url":"https://github.com/chainstacklabs/chainbench","commit_stats":null,"previous_names":[],"tags_count":38,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chainstacklabs%2Fchainbench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chainstacklabs%2Fchainbench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chainstacklabs%2Fchainbench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chainstacklabs%2Fchainbench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chainstacklabs","download_url":"https://codeload.github.com/chainstacklabs/chainbench/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247685628,"owners_count":20979085,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T21:29:25.472Z","updated_at":"2025-04-07T16:17:52.147Z","avatar_url":"https://github.com/chainstacklabs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg width=\"1200\" alt=\"Labs\" src=\"https://user-images.githubusercontent.com/99700157/213291931-5a822628-5b8a-4768-980d-65f324985d32.png\"\u003e\n\n\u003cp\u003e\n \u003ch3 align=\"center\"\u003eChainstack is the leading suite of services connecting developers with Web3 infrastructure\u003c/h3\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca target=\"_blank\" href=\"https://chainstack.com/build-better-with-ethereum/\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/soos3d/blockchain-badges/main/protocols_badges/Ethereum.svg\" /\u003e\u003c/a\u003e\u0026nbsp;  \n  \u003ca target=\"_blank\" href=\"https://chainstack.com/build-better-with-bnb-smart-chain/\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/soos3d/blockchain-badges/main/protocols_badges/BNB.svg\" /\u003e\u003c/a\u003e\u0026nbsp;\n  \u003ca target=\"_blank\" href=\"https://chainstack.com/build-better-with-polygon/\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/soos3d/blockchain-badges/main/protocols_badges/Polygon.svg\" /\u003e\u003c/a\u003e\u0026nbsp;\n  \u003ca target=\"_blank\" href=\"https://chainstack.com/build-better-with-avalanche/\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/soos3d/blockchain-badges/main/protocols_badges/Avalanche.svg\" /\u003e\u003c/a\u003e\u0026nbsp;\n  \u003ca target=\"_blank\" href=\"https://chainstack.com/build-better-with-solana/\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/soos3d/blockchain-badges/main/protocols_badges/Solana.svg\" /\u003e\u003c/a\u003e\u0026nbsp;\n\u003ca target=\"_blank\" href=\"https://chainstack.com/build-better-with-starknet/\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/soos3d/blockchain-badges/main/protocols_badges/Starknet.svg\" /\u003e\u003c/a\u003e\u0026nbsp;\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  • \u003ca target=\"_blank\" href=\"https://chainstack.com/\"\u003eHomepage\u003c/a\u003e •\n  \u003ca target=\"_blank\" href=\"https://chainstack.com/protocols/\"\u003eSupported protocols\u003c/a\u003e •\n  \u003ca target=\"_blank\" href=\"https://chainstack.com/blog/\"\u003eChainstack blog\u003c/a\u003e •\n  \u003ca target=\"_blank\" href=\"https://docs.chainstack.com/quickstart/\"\u003eChainstack docs\u003c/a\u003e •\n  \u003ca target=\"_blank\" href=\"https://docs.chainstack.com/quickstart/\"\u003eBlockchain API reference\u003c/a\u003e • \u003cbr\u003e \n  • \u003ca target=\"_blank\" href=\"https://console.chainstack.com/user/account/create\"\u003eStart for free\u003c/a\u003e •\n\u003c/p\u003e\n\n# Chainbench\n![checks status](https://github.com/chainstacklabs/chainbench/actions/workflows/checks.yml/badge.svg) \n![build status](https://github.com/chainstacklabs/chainbench/actions/workflows/python-publish.yml/badge.svg)\n![version](https://img.shields.io/pypi/v/chainbench)\n![license](https://img.shields.io/github/license/chainstacklabs/chainbench)\n\nThis project allows you to benchmark your blockchain infrastructure. It uses [Locust](https://docs.locust.io/en/stable/index.html) under the hood.\n\n## Project Details\n\nChainbench allows easy profile definition for any EVM-compatible blockchain and Solana, including partial support for Starknet. You can use both hard-coded values and real chain data for dynamic call parameters.\n\nMain features:\n- Built-in profiles for Ethereum, Binance Smart Chain, Polygon, Solana, Base, Arbitrum, Avalanche, Fantom, Gnosis, Oasis, Optimism, Ronin and Starknet, as well as some generic EVM-based profiles\n- Support for custom profiles\n- Dynamic call params generation using real chain data\n- Headless and web UI modes\n\nCheck out the [docs](docs/PROFILE.md) for more information about the profile creation.\n\n## Prerequisites\n\n- Python 3.10+\n- Poetry 1.7+ (installation instructions [here](https://python-poetry.org/docs/#installation))\n\n## Installation\n\n### Using pip\n\n```shell\npip install chainbench\n```\n\nAfter installation, you can run the tool using the following command:\n```shell\nchainbench start --help\n```\n\n### Using Poetry\n\nClone the repository:\n```shell\ngit clone git@github.com:chainstacklabs/chainbench.git\n```\n\nInstall dependencies:\n```shell\ncd chainbench \u0026\u0026 poetry install --without dev\n```\n\nWhen installing using Poetry, you can run the tool using the following command:\n```shell\npoetry run chainbench\n```\n\n## Example Usage\nAll the examples below assume that you have installed the tool using `pip`. If you installed it using `poetry`, replace `chainbench` with `poetry run chainbench`.\n\nTo list all available commands and options, run:\n```shell\nchainbench --help\n```\n`start` command is used to start a load test. To learn about the parameters and flags, run the following command:\n```shell\nchainbench start --help\n```\n\nYou may specify a method to test, for example:\n```shell\nchainbench start eth_blockNumber --users 50 --workers 2 --test-time 12h --target https://node-url --headless --autoquit \n```\nThis will run a load test with `eth_blockNumber` method calls with 2 workers, 50 users and 12 hours test time in headless mode with default test data size S.\nAfter the test is finished, the tool will automatically quit.\nIf you run the command above without '--headless' flag, it will start a web UI where you may select other methods to run tests with.\n\nTo see which methods are supported, run:\n```shell\nchainbench list methods\n```\n\nYou may also specify a profile with multiple methods and weights to use for the test instead of testing a single method. For example:\n```shell\nchainbench start --profile bsc.general --users 50 --workers 2 --test-time 12h --target https://node-url --headless --autoquit\n```\nThis will run a load test with a general BSC profile.\n\n\n### Parameters and Flags\n- `-p`, `--profile`: Specifies the profile to use for the benchmark. Available profiles can be found in the profile directory. Sample usage `-p bsc.general`\n- `-d`, `--profile-dir`: Specifies the base directory to use which contain profiles, supporting up to one-level of subdirectories.\n- `--profile-path`: Specifies the full path to the profile file to use for the benchmark. Overrides `--profile` and `--profile-dir` options.\n- `-s`, `--shape`: Specifies the shape of the load pattern. List available shapes with `chainbench list shapes`.\n- `-u`, `--users`: Sets the number of simulated users to use for the benchmark.\n- `-r`, `--spawn-rate`: Sets the spawn rate of users per second.\n- `-w`, `--workers`: Sets the number of worker threads to use for the benchmark.\n- `-t`, `--test-time`: Sets the duration of the test to run.\n- `--target`: Specifies the target blockchain node URL that the benchmark will connect to.\n- `--headless`: Runs the benchmark in headless mode, meaning that no graphical user interface (GUI) will be displayed during the test. This is useful for running the test on a remote server or when the GUI is not needed.\n- `--autoquit`: Tells the Chainbench tool to automatically quit after the test has finished. This is useful for running the benchmark in an automated environment where manual intervention is not desired.\n- `--help`: Displays the help message.\n- `--debug-trace-methods`: Enables tasks tagged with debug or trace to be executed\n- `-E`, `--exclude-tags`: Exclude tasks tagged with custom tags from the test. You may specify this option multiple times.\n- `--use-latest-blocks`: Use latest blocks for test data generation and runs a background process to update the test data with latest blocks.\n- `--size`: Specifies the test data size. Available values are XS, S, M, L, XL. Default is S.\n- `--batch`: Runs the test using batch requests. This will send multiple requests in a single batch request. The number of requests in a batch can be specified using the `--batch-size` flag. Default batch size is 10.\n\nYou may also run `chainbench start --help` for the full list of parameters and flags.\n\n### Profiles\nDefault profiles are located in the [`profile`](chainbench/profile) directory. For a tutorial on how to create custom profiles, please refer to [this document](docs/PROFILE.md).\n\nYou may use the following command to list all profiles available out of the box:\n```shell\nchainbench list profiles\n```\n\nThe `-d` or `--profile-dir` flag can be used to specify a custom directory with profiles. For example:\n```shell\nchainbench start --profile-dir /path/to/profiles --profile my-profile --users 50 --workers 2 --test-time 12h --target https://node-url --headless --autoquit\n```\nThis will run a load test using `/path/to/profiles/my-profile.py` profile.\n\nIt's possible to group profiles into directories. For example, you can create a directory called `bsc` and put all the BSC profiles there. Then you can run a load test using the following command:\n```shell\nchainbench start --profile-dir /path/to/profiles --profile bsc.my-profile --users 50 --workers 2 --test-time 12h --target https://node-url --headless --autoquit\n```\n\nChainbench will look for the profile in `/path/to/profiles/bsc/my-profile.py`. Currently, only one level of nesting is supported.\n\nThere are built-in `evm.light` and `evm.heavy` profiles for EVM-compatible chains.\n\nHere's an example of how to run a load test for Ethereum using the `evm.light` profile:\n```shell\nchainbench start --profile evm.light --users 50 --workers 2 --test-time 12h --target https://node-url --headless --autoquit\n```\n\n## Load Pattern Shapes\nLoad pattern shapes are used to define how the load will be distributed over time. You may specify the shape of the load pattern using the `-s` or `--shape` flag.\nThis is an optional flag and if not specified, the default shape will be used. The default shape is `ramp-up` which means the load will increase linearly over time at\nthe spawn-rate until the specified number of users is reached, after that it will maintain the number of users until test duration is over.\n\nOther available shapes are:\n- `step` - The load will increase in steps. `--spawn-rate` flag is required to specify the step size. The number of steps will be calculated based on `--users` divided by `--spawn-rate`. The duration of each step will be calculated based on `--test-time` divided by the number of steps.\n- `spike` - The load will run in a spike pattern. The load will ramp up to 10% of the total users for 40% of the test duration and then spike to 100% of the total users as specified by `--users` for 20% of test duration and then reduce back to 10% of total users until the test duration is over.\n\n### Test Data Size\nYou may specify the test data size using the `--size` flag. This will determine how much data is used in the test.\nTake note that larger data size will result in longer test data generation time before the test starts.\nWe plan to add an option to reuse previously generated test data in the future to avoid regenerating the same\ntest data for multiple test runs.\n\n| Size  | Blocks  |\n|-------|---------|\n| XS    | 10      |\n| S     | 100     |\n| M     | 1,000   |\n| L     | 10,000  |\n| XL    | 100,000 |\n\n\n### Test Data - Using Latest Blocks\nWhen running tests on nodes that are running in sync modes that do not have access to the full blockchain history, it is recommended to use the `--use-latest-blocks` flag.\n\nExample:\nIf the node runs in a sync mode that only keeps the last 128 blocks in history, you may run a test with the following command:\n```shell\nchainbench start --profile evm.light --users 50 --workers 2 --test-time 1h --target https://node-url --headless --autoquit --use-latest-blocks --size S\n```\n\nWith test data size S and `--use-latest-blocks` flag, the tool will generate test data using the latest 100 blocks.\nChainbench will keep the test data up to date by running a background process that will update the test data with latest blocks.\n\n### Monitors\nMonitors are separate processes that run during the test to collect or process some additional data and metrics relevant to the test.\nFor example, sync-lag-monitor will collect the latest block information from the node under test, check the timestamp and compare it to current time to calculate how much the node lags behind.\nYou may include monitors in your test by using the `-m` or `--monitor` option and specifying the name of the monitor. At the moment, monitors only work in headless mode.\n\nHere's an example:\n```shell\nchainbench start --profile evm.light --users 50 --workers 2 --test-time 12h --target https://node-url --headless --autoquit -m sync-lag-monitor\n```\n\n\n### Web UI Mode\n\nRun the following command to run a load test for BSC in UI mode. It will start a web server on port 8089. \nTarget is required to initialize the test data, however you may change the target endpoint later in the UI, along with the number of users, spawn rate and test time.\nDo take note when changing the target option that test data initialized with an Ethereum node endpoint, for example, can only be used to test other Ethereum node endpoints.\n```shell\nchainbench start --profile bsc.general --workers 1 --target https://any-working-node-endpoint.com\n```\nIf you'd like to select which profiles or more specifically which user classes to use for the test, you may start a test in Web UI Mode without specifying a profile or passing in the method argument.\n```shell\nchainbench start --workers 1 --target https://any-working-node-endpoint.com --profile-dir chainbench/profile/evm\n```\nThis will start a test with all the EVM profiles and user classes available in the UI for you to select.\n### Headless Mode\n\nIf you want to run a load test for BSC in headless mode, run the following command:\n\n```shell\nchainbench start --profile bsc.general --workers 4 --users 100 --test-time 1h --target https://node-url --headless --autoquit\n```\n\nIt will run a load test for BSC with 4 workers, 100 users and 1 hour test time.\n\nIn practice, you will probably want to run the benchmark on a remote server. Here's the example utilizing `nohup`:\n\n```shell\nnohup chainbench start --profile bsc.general --workers 4 --users 100 --test-time 1h --target https://node-url --headless --autoquit \u0026\n```\n\n## Other Commands\n### Discover Available Methods on Endpoints\nThis command will discover all available rpc methods on the specified endpoint and print them to the console. List of methods that are tested are based on the `--clients` option.\n```shell\nchainbench discover https://node-url --clients geth,erigon\n```\nTo list valid arguments for `--clients` option and the reference client version, run:\n```shell\nchainbench list clients\n```\nIf you don't specify the `--clients` option, the tool will default to Ethereum JSON-RPC Specification (eth).\n\n## License\nThis project is licensed under the [Apache 2.0 License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchainstacklabs%2Fchainbench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchainstacklabs%2Fchainbench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchainstacklabs%2Fchainbench/lists"}