https://github.com/aviggiano/property-based-testing-benchmark

Property-based testing benchmark
https://github.com/aviggiano/property-based-testing-benchmark

Last synced: 3 days ago
JSON representation

Property-based testing benchmark

• Host: GitHub
• URL: https://github.com/aviggiano/property-based-testing-benchmark
• Owner: aviggiano
• License: agpl-3.0
• Created: 2023-10-09T19:46:57.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-11-15T12:49:10.000Z (about 1 year ago)
• Last Synced: 2024-04-28T04:43:51.090Z (10 months ago)
• Language: Solidity
• Size: 215 KB
• Stars: 9
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• evm-fuzzing-resources - Property-based testing benchmark
• evm-fuzzing-resources - Property-based testing benchmark

README

        
# property-based-testing-benchmark

Property-based testing benchmark

## Description

This project aims to provide a comprehensive benchmark analysis of the performance and effectiveness of different property-based testing tools against real-world DeFi [projects](./projects/).

- [Halmos](https://github.com/a16z/halmos)

- [Foundry](https://github.com/foundry-rs/foundry/)

- [Echidna](https://github.com/crytic/echidna)

- [Medusa](https://github.com/crytic/medusa)

## Usage

Launch the benchmark [infrastructure](./infrastructure/) with Terraform:

```bash

cd infrastructure

terraform init

terraform apply

```

Run the benchmark

```bash

python3 -m benchmark producer --full-benchmark

```

Analyse the results

```bash

python3 -m benchmark analyser

```

## Project structure

- [Dockerfile](./Dockerfile): Install all referenced tools and [benchmark](./benchmark/) scripts

- [infrastructure](./infrastructure): Set of Terraform configuration files that will launch

  - SQS: queue of [job messages](./benchmark/parser.py)

  - S3: bucket of [job outputs](./benchmark/runner.py)

  - ECS: cluster of [consumer](./benchmark/consumer.py) components, responsible for reading the SQS queue and launching [runner](./benchmark/runner.py) tasks that execute the benchmark and upload results to S3

  - ECR: registry that hosts the Docker image

  - IAM: access controls for Docker containers to manage the referenced resources

  - VPC: networking required for ECS

- [benchmark](./benchmark/): Benchmark scripts

  - [producer](./benchmark/producer.py): Push job messages to the queue

  - [consumer](./benchmark/consumer.py): Pull messages from the queue and launch a new task to run the benchmark

  - [runner](./benchmark/runner.py): Run benchmark jobs, and analyse benchmark results

  - [analyser](./benchmark/analyser.py): Analyse benchmark results

## Contribute

Contributions are welcome. Feel free to open an issue or pull request.