https://github.com/aviggiano/property-based-testing-benchmark
Property-based testing benchmark
https://github.com/aviggiano/property-based-testing-benchmark
Last synced: 6 months ago
JSON representation
Property-based testing benchmark
- Host: GitHub
- URL: https://github.com/aviggiano/property-based-testing-benchmark
- Owner: aviggiano
- License: agpl-3.0
- Created: 2023-10-09T19:46:57.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2023-11-15T12:49:10.000Z (about 2 years ago)
- Last Synced: 2025-03-31T07:41:32.452Z (8 months ago)
- Language: Solidity
- Size: 215 KB
- Stars: 8
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- evm-fuzzing-resources - Property-based testing benchmark
README
# property-based-testing-benchmark
Property-based testing benchmark
## Description
This project aims to provide a comprehensive benchmark analysis of the performance and effectiveness of different property-based testing tools against real-world DeFi [projects](./projects/).
- [Halmos](https://github.com/a16z/halmos)
- [Foundry](https://github.com/foundry-rs/foundry/)
- [Echidna](https://github.com/crytic/echidna)
- [Medusa](https://github.com/crytic/medusa)
## Usage
Launch the benchmark [infrastructure](./infrastructure/) with Terraform:
```bash
cd infrastructure
terraform init
terraform apply
```
Run the benchmark
```bash
python3 -m benchmark producer --full-benchmark
```
Analyse the results
```bash
python3 -m benchmark analyser
```
## Project structure
- [Dockerfile](./Dockerfile): Install all referenced tools and [benchmark](./benchmark/) scripts
- [infrastructure](./infrastructure): Set of Terraform configuration files that will launch
- SQS: queue of [job messages](./benchmark/parser.py)
- S3: bucket of [job outputs](./benchmark/runner.py)
- ECS: cluster of [consumer](./benchmark/consumer.py) components, responsible for reading the SQS queue and launching [runner](./benchmark/runner.py) tasks that execute the benchmark and upload results to S3
- ECR: registry that hosts the Docker image
- IAM: access controls for Docker containers to manage the referenced resources
- VPC: networking required for ECS
- [benchmark](./benchmark/): Benchmark scripts
- [producer](./benchmark/producer.py): Push job messages to the queue
- [consumer](./benchmark/consumer.py): Pull messages from the queue and launch a new task to run the benchmark
- [runner](./benchmark/runner.py): Run benchmark jobs, and analyse benchmark results
- [analyser](./benchmark/analyser.py): Analyse benchmark results
## Contribute
Contributions are welcome. Feel free to open an issue or pull request.