https://github.com/charmve/benchrt
Benchmark regression testing cross-platform [Bazel] 跨平台编译运行工具，支持增量回归性能测试框架
https://github.com/charmve/benchrt
bazel cross-compile cross-compiler cross-platform embedded-systems iot
Last synced: 4 months ago
JSON representation
Benchmark regression testing cross-platform [Bazel] 跨平台编译运行工具，支持增量回归性能测试框架
- Host: GitHub
- URL: https://github.com/charmve/benchrt
- Owner: Charmve
- License: apache-2.0
- Created: 2023-02-03T16:46:19.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2024-05-27T02:45:47.000Z (over 1 year ago)
- Last Synced: 2025-07-02T20:04:31.997Z (6 months ago)
- Topics: bazel, cross-compile, cross-compiler, cross-platform, embedded-systems, iot
- Language: Perl
- Homepage:
- Size: 336 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# BenchRT
BenchRT means benchmarking regression testting or benchmark runtime in real-world device.
## Features
- QBuild: provide cross-platform run-time pipeline
- Support cross-platform, multi-platform
- Automated testing at scale
- Report automated make, alter
## Quick start
```
./scripts/start_cross_compile_docker.sh --project j5 --office suzhou
./scripts/goto_cross_compile_docker.sh --project j5
qbuild --init
qbuild --help
# eg:qbuild --benchmark j5 onboard/logging/logging_bm
```
[](https://asciinema.org/a/k0rFmkO8EZCwn6hj8KEzZuCzL)
演示链接：https://asciinema.org/a/k0rFmkO8EZCwn6hj8KEzZuCzL
## Usage
### qbuild
```bash
charmve@SZ-zhangwei:/charmve$ qbuild --help
🚀 Welcome to use QBuild!
USAGE:
qbuild [options] []
VERSION:
1.0.0
OPTIONS:
-i, --init Make qbuild init
-b, --build Build module, include unit test, benchmark single
module and mutil-module
-it, --install Make module package
-c, --connect Connect to target platform easy, include real env and virtual env
-d, --deploy Deploy module binary
-r, --run Run module binary in mutil-platform
-u, --update Update the sysroot by manaul
-f, --format Format code in X-Comipler docker
-p, --pull Pull file from arm-platform to local, like module
library or config file
-t, --test Test module (only for qbuild SDEer)
-ut, --unittest Debug unit test
-bm, --benchmark Run benchmark to get report
-pf, --perf Perf module performance
-co, --coverage Give the coverage report in target platform
-sc, --sonarqube Source code static analysis and security scan
-is, --issues Pull issues about Qbuild to Onboard Infra
-cl, --clean Clean QBuild cache
-v, --version QBuild's version
-h, --help Show this message and exit
Run 'qbuild COMMAND' for more information on a command.
Visit https://qcraft.feishu.cn/docx/Vo6GdVCDqow0v5xX5RkcxBTXnQb to get more information and push issues.
```
#### 使用方法：
进入 X-Compiler docker 容器中，直接qbuild命令可以看到整个工具的usage，都有交互提示。
1. Update to latest master
2. ./scripts/start_cross_compile_docker.sh --project j5 --office suzhou
3. ./scripts/goto_cross_compile_docker.sh --project j5
4. export PATH=${PATH}:/qcraft/qbuild
5. qbuild --init
6. Find qbuild usages qbuild --help
7. 命令输入每一个都会有检查和使用提示，如：qbuild --run、qbuild --build
#### 结果展示
onboard/lite 路径下 单元测试结果
#### TO-DO
- --perf 模块级别的profile性能分析
- --install 打包方式
- 需要稳定的台架资源
- 命令自动补全
### bench-rt
```bash
cd BenchRT
nohup python3 -m http.server >>/dev/null &
export PATH=$PATH:$(pwd)/google-cloud-sdk/bin
bazel run :benchmark -- --bazel_commits=fffc26b5cc1bbe6c977af9971ed21e2e3d275d28,25be21130ba774e9f02cc39a010aafe64a3ab245 --project_source=/charmve/ --project_commits=6dd9685b9e --data_directory=/tmp/bazel-bench-data --verbose --platform=x86 --project_label=dev-test --collect_profile=True --aggregate_json_profiles=True -- run --verbose_failures //qbuild/examples/helloworld:helloworld
bazel run :benchmark -- --bazel_commits=fffc26b5cc1bbe6c977af9971ed21e2e3d275d28,25be21130ba774e9f02cc39a010aafe64a3ab245 --project_source=/charmve/ --project_commits=6dd9685b9e --platform=x86 --project_label=dev-test -- run --verbose_failures //onboard/math:vec_bm
bazel run :benchmark -- --bazel_commits=fffc26b5cc1bbe6c977af9971ed21e2e3d275d28,25be21130ba774e9f02cc39a010aafe64a3ab245 --project_source=/charmve/ --project_commits=6dd9685b9e --platform=$1 --project_label=dev-test -- run --verbose_failures onboard/math/fast_math_bm
bazel run report:generate_report -- --storage_bucket=0.0.0.0:8000/bazel-bench/bazel-bench-data --project=bazel-bench-test
```
结果展示：
- ci 自动化评测
- x86-有对应bm：
- x86-无对应bm：
pre-run一次，再统计连续跑5次作为最终结果。
收集汇总所有的json-profile文件：
``bazel run :benchmark -- --bazel_commits=fffc26b5cc1bbe6c977af9971ed21e2e3d275d28,25be21130ba774e9f02cc39a010aafe64a3ab245 --project_source=/qcraft/ --project_commits=6dd9685b9e --platform=x86 --project_label=dev-test --collect_profile=True --aggregate_json_profiles=True -- run --verbose_failures //qbuild/examples/hiqcraft:hiqcraft``
- j5-有对应bm：
qbuild --run
j5-无对应bm：
## Profiling
[Build performance metrics](https://blog.bazel.build/2022/11/15/build-performance-metrics.html)
## Setup runtime environment
docs: docs/ARM64编译环境搭建-以J5为例.docx
### multi-platform switch 'goto x'
```
sudo cat >> "$HOME/.bashrc" << EOF
function goto() {
platform=$1
cd ~/charmve
existing_running_docker=("$(docker ps --filter name="charmve")")
if [ $platform == j5 ] || [ $platform == orin ]; then
if [[ " ${existing_running_docker[*]} " =~ cross_compile_[j5,orin] ]]; then
./scripts/goto_cross_compile_docker.sh --project $platform
else
./scripts/start_cross_compile_docker.sh --project $platform --office suzhou
./scripts/goto_cross_compile_docker.sh --project $platform
fi
elif [ $platform == x86 ]; then
if [[ " ${existing_running_docker[*]} " =~ dev_charmve ]]; then
./scripts/goto_dev_docker.sh
else
./scripts/start_dev_docker.sh --office suzhou
./scripts/goto_dev_docker.sh
fi
elif [ $platform == j5_v ]; then
docker exec -it j5 bash
else
echo "goto j5/j5_v/orin/x86"
# exit 0
fi
}
EOF
source $HOME/.bashrc
```
### setup virtual hb-j5 environment
``
docker build -t charmve/j5-bionic-qemu-20221230_0011:v1 -f dev/j5-qemu.aarch64.dockerfile
``