{"id":18813997,"url":"https://github.com/ackee-blockchain/python-testing-frameworks-benchmark","last_synced_at":"2025-04-13T21:26:27.218Z","repository":{"id":165697515,"uuid":"620913628","full_name":"Ackee-Blockchain/python-testing-frameworks-benchmark","owner":"Ackee-Blockchain","description":"This repository contains code and measured data for evaluation of testing frameworks for Ethereum Blockchain Applications","archived":false,"fork":false,"pushed_at":"2023-11-28T16:32:21.000Z","size":1875,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-27T11:43:42.910Z","etag":null,"topics":["blockchain","ethereum","frameworks","testing"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ackee-Blockchain.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-03-29T16:06:29.000Z","updated_at":"2023-12-07T10:11:02.000Z","dependencies_parsed_at":null,"dependency_job_id":"492f2875-5258-4027-a6b3-47f873852c98","html_url":"https://github.com/Ackee-Blockchain/python-testing-frameworks-benchmark","commit_stats":null,"previous_names":["ackee-blockchain/python-testing-frameworks-benchmark"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ackee-Blockchain%2Fpython-testing-frameworks-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ackee-Blockchain%2Fpython-testing-frameworks-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ackee-Blockchain%2Fpython-testing-frameworks-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ackee-Blockchain%2Fpython-testing-frameworks-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ackee-Blockchain","download_url":"https://codeload.github.com/Ackee-Blockchain/python-testing-frameworks-benchmark/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248784252,"owners_count":21161084,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blockchain","ethereum","frameworks","testing"],"created_at":"2024-11-07T23:39:02.618Z","updated_at":"2025-04-13T21:26:27.182Z","avatar_url":"https://github.com/Ackee-Blockchain.png","language":"TypeScript","readme":"# Intro\n\nThis repository contains code and measured data for evaluation of testing frameworks for Ethereum Blockchain Applications.\n\nIn this repository you will find the following:\n* `test_projects.py`: Python script to run the test projects and measure times\n* `test_tests_config.json`: JSON file containing the configuration for the test projects\n* `process_results.py` Python script to process the measured times\n* `test_results.csv`: CSV file containing the measured times\n* `processed_results.csv` CSV file containing the processed results of measured times\n* `v3_core`: Directory with rewritten tests for [v3-core](https://github.com/Uniswap/v3-core)\n* `README.md`: This readme\n\n# Methodology\n\nTesting was done with 4 different testing frameworks, namely Wake, Brownie and Ape which are frameworks written in Python and Hardhat+Ethers.js which was used in the original tests.\nThree different development chains, namely Anvil, Ganache and Hardhat.\nAs for the tests, popular Uniswap-v3 project and its [v3-core](https://github.com/Uniswap/v3-core) was chosen and it's tests partly rewritten and repurposed for each of the python resulting in 271 tests.\n\nTests were conducted on a VM n2d-standard-4 instance in Google Cloud, equipped with four cores of AMD Rome, 16 GB RAM, and an HDD disk. The system ran a Debian OS with kernel version 5.10.162-1 and Python 3.10.11. All tests were executed under similar load conditions on the virtual machine.\n\nAll Python frameworks have their requirements files in `v3_core` directory. As for the TypeScript tests, packages in `package.json` were installed.\n\n# Results\n\nThe execution times **in seconds** of the tests are shown in the following table in format: **mean (standard deviation)**. Tests were executed and **measured 200 times**.\n\n| /       | Brownie      | Ape          | Wake         | Hardhat \u0026 Ethers.js |\n|---------|--------------|--------------|--------------|---------------------|\n| Anvil   | 34.80 (1.31) | 53.27 (1.22) | 3.37 (0.05)  | 10.96 (0.40)        |\n| Ganache | 51.48 (1.41) | 72.72 (2.09) | 15.78 (0.22) | 118.71 (1.69)       |\n| Hardhat | 51.62 (2.43) | 72.42 (1.80) | 19.69 (0.15) | 17.47 (0.17)        |\n\n\n# Necessary modifications\n\nIt was necessary to modify Brownie's `network\\rpc\\anvil.py` to allow us to specify two additional arguments for Anvil and also to fix an issue where PIPE output was not being read correctly thus resulting in hangs when deploying large contracts. The modified file is included in this repository as `modified_anvil.py` in the `v3_core` directory.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fackee-blockchain%2Fpython-testing-frameworks-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fackee-blockchain%2Fpython-testing-frameworks-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fackee-blockchain%2Fpython-testing-frameworks-benchmark/lists"}