{"id":16562822,"url":"https://github.com/charmve/benchrt","last_synced_at":"2026-05-16T08:04:07.372Z","repository":{"id":106309792,"uuid":"597109988","full_name":"Charmve/BenchRT","owner":"Charmve","description":"Benchmark regression testing cross-platform [Bazel] 跨平台编译运行工具，支持增量回归性能测试框架","archived":false,"fork":false,"pushed_at":"2024-05-27T02:45:47.000Z","size":344,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-07-02T20:04:31.997Z","etag":null,"topics":["bazel","cross-compile","cross-compiler","cross-platform","embedded-systems","iot"],"latest_commit_sha":null,"homepage":"","language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Charmve.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-02-03T16:46:19.000Z","updated_at":"2025-04-04T12:59:08.000Z","dependencies_parsed_at":"2025-04-22T03:06:16.308Z","dependency_job_id":null,"html_url":"https://github.com/Charmve/BenchRT","commit_stats":{"total_commits":13,"total_committers":2,"mean_commits":6.5,"dds":"0.23076923076923073","last_synced_commit":"9bd8879d1b043b26dfed00c7c51d4bf48cbaeaee"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Charmve/BenchRT","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Charmve%2FBenchRT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Charmve%2FBenchRT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Charmve%2FBenchRT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Charmve%2FBenchRT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Charmve","download_url":"https://codeload.github.com/Charmve/BenchRT/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Charmve%2FBenchRT/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263208054,"owners_count":23430675,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bazel","cross-compile","cross-compiler","cross-platform","embedded-systems","iot"],"created_at":"2024-10-11T20:37:19.163Z","updated_at":"2026-05-16T08:04:07.293Z","avatar_url":"https://github.com/Charmve.png","language":"Perl","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BenchRT\n\nBenchRT means benchmarking regression testting or benchmark runtime in real-world device.\n\n## Features\n\n- QBuild: provide cross-platform run-time pipeline\n- Support cross-platform, multi-platform\n- Automated testing at scale\n- Report automated make, alter\n\n## Quick start\n\n```\n./scripts/start_cross_compile_docker.sh --project j5 --office suzhou\n./scripts/goto_cross_compile_docker.sh --project j5\n\nqbuild --init\n\nqbuild --help\n# eg:qbuild --benchmark j5 onboard/logging/logging_bm\n```\n\n[![image](https://user-images.githubusercontent.com/29084184/219936570-c25ed0d4-f588-4e43-bcb7-f5597e7e5ba0.png)](https://asciinema.org/a/k0rFmkO8EZCwn6hj8KEzZuCzL)\n\n演示链接：https://asciinema.org/a/k0rFmkO8EZCwn6hj8KEzZuCzL\n\n## Usage\n\n### qbuild\n\n```bash\ncharmve@SZ-zhangwei:/charmve$ qbuild --help\n🚀 Welcome to use QBuild!\n\nUSAGE:\n    qbuild \u003ccommand\u003e [options] [\u003carguments...\u003e]\n\nVERSION:\n    1.0.0\n\nOPTIONS:\n    -i, --init              Make qbuild init\n    -b, --build             Build module, include unit test, benchmark single\n                            module and mutil-module\n    -it, --install          Make module package\n    -c, --connect           Connect to target platform easy, include real env and virtual env\n    -d, --deploy            Deploy module binary\n    -r, --run               Run module binary in mutil-platform\n    -u, --update            Update the sysroot by manaul\n    -f, --format            Format code in X-Comipler docker\n    -p, --pull              Pull file from arm-platform to local, like module \n                            library or config file\n    -t, --test              Test module (only for qbuild SDEer)\n    -ut, --unittest         Debug unit test\n    -bm, --benchmark        Run benchmark to get report\n    -pf, --perf             Perf module performance\n    -co, --coverage         Give the coverage report in target platform\n    -sc, --sonarqube        Source code static analysis and security scan\n    -is, --issues           Pull issues about Qbuild to Onboard Infra\n    -cl, --clean            Clean QBuild cache\n    -v, --version           QBuild's version\n    -h, --help              Show this message and exit\n\nRun 'qbuild COMMAND' for more information on a command.\n\nVisit https://qcraft.feishu.cn/docx/Vo6GdVCDqow0v5xX5RkcxBTXnQb to get more information and push issues.\n```\n\n#### 使用方法：\n进入 X-Compiler docker 容器中，直接qbuild命令可以看到整个工具的usage，都有交互提示。\n\n1. Update to latest master \n2. ./scripts/start_cross_compile_docker.sh --project j5 --office suzhou\n3. ./scripts/goto_cross_compile_docker.sh --project j5\n4. export PATH=${PATH}:/qcraft/qbuild\n5. qbuild --init\n6. Find qbuild usages qbuild --help\n7. 命令输入每一个都会有检查和使用提示，如：qbuild --run、qbuild --build \n\n#### 结果展示\n\nonboard/lite 路径下 单元测试结果\n\n![image](https://user-images.githubusercontent.com/29084184/219936651-ed6d306e-b789-44d5-9a48-0fe31f545ff7.png)\n\n#### TO-DO\n- --perf 模块级别的profile性能分析\n- --install 打包方式\n- 需要稳定的台架资源\n- 命令自动补全\n\n### bench-rt\n\n```bash\ncd BenchRT\nnohup python3 -m http.server \u003e\u003e/dev/null \u0026\nexport PATH=$PATH:$(pwd)/google-cloud-sdk/bin\n\nbazel run :benchmark -- --bazel_commits=fffc26b5cc1bbe6c977af9971ed21e2e3d275d28,25be21130ba774e9f02cc39a010aafe64a3ab245 --project_source=/charmve/ --project_commits=6dd9685b9e --data_directory=/tmp/bazel-bench-data --verbose --platform=x86 --project_label=dev-test  --collect_profile=True --aggregate_json_profiles=True -- run  --verbose_failures //qbuild/examples/helloworld:helloworld\n\nbazel run :benchmark -- --bazel_commits=fffc26b5cc1bbe6c977af9971ed21e2e3d275d28,25be21130ba774e9f02cc39a010aafe64a3ab245 --project_source=/charmve/ --project_commits=6dd9685b9e --platform=x86 --project_label=dev-test  -- run  --verbose_failures //onboard/math:vec_bm\n\nbazel run :benchmark -- --bazel_commits=fffc26b5cc1bbe6c977af9971ed21e2e3d275d28,25be21130ba774e9f02cc39a010aafe64a3ab245 --project_source=/charmve/ --project_commits=6dd9685b9e --platform=$1 --project_label=dev-test  -- run  --verbose_failures onboard/math/fast_math_bm\n\n\nbazel run report:generate_report --  --storage_bucket=0.0.0.0:8000/bazel-bench/bazel-bench-data --project=bazel-bench-test\n\n```\n\n结果展示：\n\n- ci 自动化评测\n\n![image](https://user-images.githubusercontent.com/29084184/231991735-4e4a7263-5b01-465d-9cfc-7fcde9573a58.png)\n\n\n- x86-有对应bm：\n\n![image](https://user-images.githubusercontent.com/29084184/219936292-56333dba-14fb-4436-a175-f4bb97c097d0.png)\n\n![image](https://user-images.githubusercontent.com/29084184/219936285-5ce45b39-e1f3-47e4-b396-c31aeee3df47.png)\n\n\n- x86-无对应bm：\n\npre-run一次，再统计连续跑5次作为最终结果。\n\n![image](https://user-images.githubusercontent.com/29084184/219936283-80244727-c66d-434f-99d1-981da7f57098.png)\n\n![image](https://user-images.githubusercontent.com/29084184/219936274-b9181d66-78ec-4514-8ec3-1c80baca30c9.png)\n\n\n收集汇总所有的json-profile文件：\n``bazel run :benchmark -- --bazel_commits=fffc26b5cc1bbe6c977af9971ed21e2e3d275d28,25be21130ba774e9f02cc39a010aafe64a3ab245 --project_source=/qcraft/ --project_commits=6dd9685b9e --platform=x86 --project_label=dev-test  --collect_profile=True --aggregate_json_profiles=True -- run  --verbose_failures //qbuild/examples/hiqcraft:hiqcraft``\n\n- j5-有对应bm：\n\nqbuild --run \u003cplatform\u003e\n\n![image](https://user-images.githubusercontent.com/29084184/219936266-7496818d-768f-48c4-b896-f1591ded519e.png)\n\n\nj5-无对应bm：\n\n![image](https://user-images.githubusercontent.com/29084184/219936257-0c61d4a9-c718-4cf1-957f-bef3c07d7550.png)\n\n\n## Profiling\n\n[Build performance metrics](https://blog.bazel.build/2022/11/15/build-performance-metrics.html)\n\n![image](https://user-images.githubusercontent.com/29084184/219936441-85e68619-8f29-433e-90d3-c0f8d457ef8f.png)\n\n\n## Setup runtime environment\n\ndocs: docs/ARM64编译环境搭建-以J5为例.docx\n\n\n### multi-platform switch 'goto x' \n```\nsudo cat \u003e\u003e \"$HOME/.bashrc\" \u003c\u003c EOF\n\nfunction goto() {\n platform=$1\n cd ~/charmve\n \n existing_running_docker=(\"$(docker ps --filter name=\"charmve\")\")\n if [ $platform == j5 ] || [ $platform == orin ]; then\n    if [[ \" ${existing_running_docker[*]} \" =~ cross_compile_[j5,orin] ]]; then\n        ./scripts/goto_cross_compile_docker.sh --project $platform\n    else\n        ./scripts/start_cross_compile_docker.sh --project $platform --office suzhou\n        ./scripts/goto_cross_compile_docker.sh --project $platform\n    fi\n elif [ $platform == x86 ]; then\n    if [[ \" ${existing_running_docker[*]} \" =~ dev_charmve ]]; then\n        ./scripts/goto_dev_docker.sh\n    else\n        ./scripts/start_dev_docker.sh --office suzhou\n        ./scripts/goto_dev_docker.sh\n    fi\n elif [ $platform == j5_v ]; then\n    docker exec -it j5 bash\n else\n    echo \"goto j5/j5_v/orin/x86\"\n    # exit 0\n fi\n}\n\nEOF\n\nsource $HOME/.bashrc\n\n```\n\n### setup virtual hb-j5 environment\n \n``\ndocker build -t charmve/j5-bionic-qemu-20221230_0011:v1 -f dev/j5-qemu.aarch64.dockerfile \n``\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharmve%2Fbenchrt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcharmve%2Fbenchrt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharmve%2Fbenchrt/lists"}