https://github.com/scalar-labs/kelpie-lamb
Benchmark tool for Scalar DL with Kelpie
https://github.com/scalar-labs/kelpie-lamb
Last synced: about 2 months ago
JSON representation
Benchmark tool for Scalar DL with Kelpie
- Host: GitHub
- URL: https://github.com/scalar-labs/kelpie-lamb
- Owner: scalar-labs
- License: apache-2.0
- Created: 2021-01-08T02:13:07.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2021-01-14T02:07:26.000Z (over 4 years ago)
- Last Synced: 2025-01-12T02:56:42.429Z (3 months ago)
- Language: Java
- Size: 77.1 KB
- Stars: 2
- Watchers: 13
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Kelpie Lamb
Kelpie Lamb is a Kelpie extension for benchmarking of Scalar DL contracts.
## Build if needed
```console
$ ./gradlew shadowJar
```
## Prerequisite
You first have to set up your Scalar DL environment before benchmarking.
You can set up the environment with [scalar-terraform](https://github.com/scalar-labs/scalar-terraform) easily.
## How to run a test
1. Make Lamb configuration file. See [lamb_config.toml](./lamb_config.toml) as a sample.
2. Make a contract configuration file to specify your contract
- Refer to [Contract configuration](#Contract-configuration)
- [sample_contract_config.json](./sample_contract_config.json) is a sample configuration
3. Make a variable configuration file to configure variables defined in the contract configuration file
- Refer to [Variable configuration](#Variable-configuration)
- [sample_variable_config.json](./sample_variable_config.json_config.json) is a sample configuration
4. Run a test with Kelpie
```console
$ ${KELPIE}/bin/kelpie --config
```
### With docker
#### Build if needed
```console
$ docker build . -t ghcr.io/scalar-labs/kelpie-lamb
```
#### Pull from the GitHub Container Registry
```console
$ docker pull ghcr.io/scalar-labs/kelpie-lamb
```
#### Run a test
```console
$ ./docker_run.sh -l -c -v -p -t
```
- Your contract classpaths in the contract configuration file should be under `/lamb` directory because the script places your contracts to `/lamb` directory on docker, as seen below, for example.
```json
"sample_contract.Balance": {
"class_file": "/lamb/Balance.class",
"properties": {},
"arguments": {
"asset_id": "_id_"
}
}
```
- You don't have to change the paths of the contract configuration file and the variable configuration file in the default [lamb_config.toml](./lamb_config.toml)
## Benchmark configuration
You can configure a benchmark job by a toml file just like usual Kelpie applications.
As you can see in the sample configuration file [lamb_config.toml](./lamb_config.toml), it must specify pre-defined `LambPreparer`, `LambProcessor`, and `LambReporter` modules to make benchmarking Lamb-based. You can refer to Kelpie's [README](https://github.com/scalar-labs/kelpie) about `[modules]`, `[common]` and `[stats]` table.
### [benchmark_config]
A population contract, a target contract and your contract information are specified in this table. They are used for executing contracts in your benchmark.
- `contract_config`: A file for the contract configuration
- `variable_config`: A file for the contracts' variables
- `population_contract`: The name of a contract that populates initial assets before benchmarking
- `target_contract`: The name of a contract which you want to benchmark
- `populateion_concurrency`: The number of concurrencies to execute the population contract. The default value is 1.
- `num_populateions`: The number of times to execute the population contract per thread. The number of total execution times is the product of `population_concurrency` and `num_populations`. The default value is 1.
### [client_config]
They are information for your Scalar DL server.
- `dl_server`: The address of Scalar DL server
- `certificate`: A certificate file that will be registered to the server to register and execute contracts
- `private_key`: The corresponding private key with the certificate to handle your contracts
## Contract configuration
The contract configuration file configures your contracts by the class file location, properties and arguments.
- `class_file`: The contract class file
- `properties`: The properties of the contract which are set when the contract is registered. You don't use a variable for the properties.
- `arguments`: The arguments for the contracts. If the name of a value is a "variable" like `_var_`, the value can be dynamically set by Lamb.
## Variable configuration
If the name of a value starts with and ends with '_' in the contract configuration file, the value is a variable. Lamb set a value for each contract execution.
The variable configuration file configures the variables. Each variable can be configured independently. It is specified by the contract name and the variable name. The following parameters can be set for each variable.
- `type`: The type of a value. `INT`, `BIGINT`, `DOUBLE`, `BOOLEAN`, `STRING` are supported.
- `pattern`: The pattern of a value to choose the specific values. `SEQUENTIAL`, `RANDOM` or `TIMESTAMP` can be set.
- `SEQUENTIAL` chooses a value from a defined list in order or increments/decrements a value from the start value to the end
- `RANDOM` chooses a value from a defined list randomly or chooses a value between the start value and the end
- `TIMESTAMP` sets the current time to a value
- `list`: A value is chosen from the list when `SEQUENTIAL` and `RANDOM` pattern
- `start`: This specifies the start of the range for `SEQUENTIAL` and `RANDOM` pattern. It should be a number. (Inclusive)
- `end`: This specifies the end of the range for `SEQUENTIAL` and `RANDOM` pattern. It should be a number. (Exclusive)
- `length`: The length of a random string. This is used when the type is `STRING` and the pattern is `RANDOM`.
The following JSON is an example of a variable configuration in the case where your contract name is `my.contract.Transfer` and your variable name `amount`, and you want to give a random integer value from 100 to 1000 when the contract execution.
```json
"my.contract.Transfer": {
"amount": {
"type": "INT",
"pattern": "RANDOM",
"start": 100,
"end": 1000
}
}
```