https://github.com/stacks-archive/clarity-benchmarking
https://github.com/stacks-archive/clarity-benchmarking
Last synced: about 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/stacks-archive/clarity-benchmarking
- Owner: stacks-network
- License: mit
- Created: 2020-12-23T00:21:50.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2024-03-28T18:19:51.000Z (over 2 years ago)
- Last Synced: 2024-04-14T11:50:54.567Z (over 2 years ago)
- Language: Jupyter Notebook
- Size: 38.3 MB
- Stars: 1
- Watchers: 13
- Forks: 5
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# Clarity Native Function Benchmarking
Benchmarking suite to determine constants for usage in the Clarity Cost Functions boot contract
## Usage
### Running the benchmark suite
Running the benchmarking suite can be done via the cargo `bench` command:
```
cargo bench
```
Benchmarking results will be outputted to the `target/criterion/` directory.
### Running regression analysis
Once the benchmarking results have been collected, analysis scripts are
used to compute regressions on the data, outputting estimated functions
for each cost function. These estimated functions have units of
`nanoseconds per 100 executions` in the `y` dimension, and the input
unit for the Clarity cost function in the `x` dimension.
The analysis script requires `SciPy`, so for convenience, a Dockerfile is
included to perform analysis.
Via docker:
```
docker build -f analysis/analysis.Dockerfile --build-arg criterion_dir="latest-data/criterion" -o /tmp/analysis-output/ .
```
The `criterion_dir` optional argument tells the analysis script where in
the current directory to look for the benchmarking outputs. `latest-data`
in this repository contains the data from the execution used to produce
the current proposal data.
This will output `cost_constants.csv` to `/tmp/analysis-output` and graphs
of the analyzed data and regression.
### Translating regression analysis into proposed costs
Once the regression is performed, the proposed cost functions need to
scale those estimates into the correct dimensions for Clarity costs.
The runtime block limit in the Stacks network is `5e9` (unitless), and
the goal of the current proposal is that this should correspond to 30
seconds or `3e10` nanoseconds. The `proposal/make_cost_functions.py`
script performs that scaling and formats output appropriate for a new
costs contract (i.e., Clarity language formatted) and for inclusion in
markdown proposals (a markdown-formatted table).
To run this script:
```
cd ./proposal
cp /tmp/analysis-output/cost_constants.csv ./estimated_constants.csv
python ./make_cost_functions.py
```
This will output to `new_costs.clar` and `updates_table.md`