https://github.com/nethermindeth/blockifier-tester
A testing utility to run juno with our version of the blockifier that uses cairo native, and compare it to results from mainnet or from juno using the unmodified blockifier
https://github.com/nethermindeth/blockifier-tester
nubia
Last synced: about 2 months ago
JSON representation
A testing utility to run juno with our version of the blockifier that uses cairo native, and compare it to results from mainnet or from juno using the unmodified blockifier
- Host: GitHub
- URL: https://github.com/nethermindeth/blockifier-tester
- Owner: NethermindEth
- License: apache-2.0
- Created: 2024-06-07T07:23:58.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2025-03-07T17:07:00.000Z (3 months ago)
- Last Synced: 2025-03-22T00:33:59.014Z (2 months ago)
- Topics: nubia
- Language: Rust
- Homepage:
- Size: 169 MB
- Stars: 1
- Watchers: 4
- Forks: 2
- Open Issues: 12
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Blockifier Tester
> ⚠️ **NOTE**: This project goal is to test software which has not yet reached production stage which means setting it up might not be as easy as it should. Please follow the README thoroughly since we tried to add every possible shortcomming to it. If you see there is something missing here that might help with anything please open an issue and we will fix it asap.
This is a tool created for testing the [Blockifier integrated with Cairo Native](https://github.com/NethermindEth/blockifier). It is still a work in progress and currenty it is meant to be used only to assert that _natively compiled contracts_ behaves the same way as _casm compiled contract_.
## Terminology
**Native Juno**: Juno instance running with [Native Blockifier](https://github.com/NethermindEth/blockifier), that is, executing Cairo transactions natively by compiling them first with [Cairo Native](https://github.com/lambdaclass/cairo_native).
**Base Juno**: Juno instance running using the "normal" [Blockifier](https://github.com/starkware-libs/blockifier) or executing Cairo transactions the same way as always, with the VM.
## What it does
The tool takes either a single block or a block range\* (see [Usage](#usage)).
For each block it will:
1. attempt to trace the block with Native Juno
2. if the trace had no failures\*\* then
3. the block will be traced with Base Juno and
4. a comparison between the two results will be written in `./results-/trace-`.
Otherwise, if there was a failure, the block will be scanned (using binary search) until the faulty transaction is found. The report will be written in `./results-/block-`.
Please note that everything in `./results-/` is being tracked using [Git LFS](#git-lfs) to remove noise from the codebase.
> \*Blocks are sorted in ascending order of how many transactions they have to avoid having to run many long RPC calls before we can get any results.
> \*\*A failure in this case is defined as _any_ of the following:
>
> 1. Juno crashing
> 2. The block is not found (this likely means your Juno database did not have the block)
## Setup
### Dependencies (Juno: base and native)
To get your base version of Juno you need to first clone the [repo](https://github.com/NethermindEth/juno) and build it via `make juno`. Be sure to install all needed dependencies first, which are specified in the that same repository.
Then, to obtain the native version, clone the project again, _switch to `native2.x.x-blockifier` branch_ and recompile. If you haven't compiled Cairo Native before you may face many compilation issues. We suggest you clone [Cairo Native](https://github.com/lambdaclass/cairo_native) and compile it separately first _(be sure to be using the same version as in `native2.x.x-blockifier`)_. After both projects are compiled, make sure you have Cairo Native runtime library in your environment which is **essential** for running Ahead of Time Compiled Cairo.
```
export CAIRO_NATIVE_RUNTIME_LIBRARY=//cairo_native/target/release/libcairo_native_runtime.a
```
### Getting network
Finally, Juno must be in sync with the latest blockchain history for whichever network you're using. To achieve this you can either:
1. (recommended) Manually download a snapshot from [here](https://github.com/NethermindEth/juno?tab=readme-ov-file#-snapshots) (direct links: [Starknet](https://juno-snapshots.nethermind.dev/files/mainnet/latest), [Sepolia](https://juno-snapshots.nethermind.dev/files/sepolia/latest)). Be sure that the snapshot you are downloading is recent enough.
2. Sync Juno manually (for Starknet this will take around 4 to 5 days in optimal conditions)
### Config
In the `config.toml` located at the project root set the following variables\*:
```toml
juno_path = ""
juno_native_path = ""
juno_mainnet_database_path = "" # correlates to `--db-path` argument passed to Juno
juno_sepolia_database_path = "" # correlates to `--db-path` argument passed to Juno
```
It is recommended that you use absolute paths and avoid `$HOME` and `~`
Example `config.toml`:
```toml
juno_path = "/home/pwhite/repos/juno/build/juno"
juno_native_path = "/home/pwhite/repos/native_juno/build/juno"
juno_mainnet_database_path = "/home/pwhite/snapshots/juno_mainnet"
juno_sepolia_database_path = "/home/pwhite/snapshots/juno_sepolia"
```
### Git LFS
To mantain all results in the same repo and use github as a synced db we used Git LFS. It is used to keep track of all files in the `./results-/` directory. Follow install instructions [here](https://git-lfs.com/).
## Usage
Once setup is complete, build the project with `cargo build`. The tool presents two commands:
- _**block** ``_ which traces and perform comparisons between base and native juno over a single block.
- _**range** `` ``_ which does the same but over a range of blocks from inclusive `` to exclusive ``. Note that _(currently)_ blocks are sorted in ascending order by their amount of transactions.
Execute them with:
```bash
target/cprof/juno_compare_traces block 610508
```
or
```bash
target/cprof/juno_compare_traces range 610508 611000
```
Once the tester has traced a block, it will store the output in `./results-/.json` and it won't re-do it again unless explicetly told with the `redo-comp` flag.
The tool has many other options like this, please run the CLI with `--help` to get detailed information of all the commands and options.
```bash
target/cprof/juno_compare_traces --help
```
> **cprof** stands for compilation profile. It is usually debug or release
> There are some issues with pathing currently. It's advised to run the binary from the project root.
### Logging
The tool will log it's execution. Currently it defaults to `debug`, but you can set any of the other logging profiles (i.e. `info`, `warn` and `error`) setting the `LOG_LEVEL` variable.
```
LOG_LEVEL=info juno_compare_traces range 610508 611000
```
## Troubleshooting
_Problem_: A block fails on Juno Native when I don't expect it to.
_Suggestion_: Check `juno_out.log` to see what the actual failure was. If the failure was that the block was not found, check `juno_database_path` in your `Config.toml` and make sure it's pointing to a database path that has that block.