{"id":13745992,"url":"https://github.com/ivafanas/sltbench","last_synced_at":"2025-05-09T06:31:15.717Z","repository":{"id":51268076,"uuid":"61478446","full_name":"ivafanas/sltbench","owner":"ivafanas","description":"C++ benchmark tool. Practical, stable and fast performance testing framework.","archived":false,"fork":false,"pushed_at":"2023-01-27T03:26:01.000Z","size":349,"stargazers_count":149,"open_issues_count":11,"forks_count":10,"subscribers_count":17,"default_branch":"master","last_synced_at":"2024-05-22T16:33:10.313Z","etag":null,"topics":["benchmark","benchmark-framework","benchmarking","cplusplus","cpp","cpp-library","open-source","performance","test-framework","testing","testing-tools"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ivafanas.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-06-19T12:12:16.000Z","updated_at":"2024-04-14T13:16:13.000Z","dependencies_parsed_at":"2023-02-15T04:46:26.575Z","dependency_job_id":null,"html_url":"https://github.com/ivafanas/sltbench","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivafanas%2Fsltbench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivafanas%2Fsltbench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivafanas%2Fsltbench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivafanas%2Fsltbench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ivafanas","download_url":"https://codeload.github.com/ivafanas/sltbench/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253206066,"owners_count":21871158,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","benchmark-framework","benchmarking","cplusplus","cpp","cpp-library","open-source","performance","test-framework","testing","testing-tools"],"created_at":"2024-08-03T06:00:42.368Z","updated_at":"2025-05-09T06:31:15.098Z","avatar_url":"https://github.com/ivafanas.png","language":"C++","readme":"# sltbench\n\n[![Build Status](https://travis-ci.org/ivafanas/sltbench.svg)](https://travis-ci.org/ivafanas/sltbench)\n[![Build status](https://ci.appveyor.com/api/projects/status/s0q3gnqprrqsahda?svg=true)](https://ci.appveyor.com/project/ivafanas/sltbench)\n\nC++ benchmark tool. Practical, stable and fast performance testing framework.\n- practical - came up from daily production tasks. Usability matters\n- stable - correctness and reproducibility is a goal\n- fast - if results are stable, they should be produced as soon as possible\n\nThis tool is trained to gain about 5% measure stability (depends on function,\ninput and context). Of course, it cannot strongly guarantee 5% stability (some\nfunctions are unstable itself, nothing helps), but it tends to.\n\n\n# Why sltbench\n\nThe closest and the most famous analogue is googlebench - microbenchmarking\nlibrary from Google engineers.\n\nKey features of `sltbench` are:\n* sltbench is about *4.7x times faster* than googlebench (test execution time).\n* sltbench produces results with less timing error, which matters for\nperformance results reproducibility. About 5x times improvement for testing\ndataset.\n* Supports correct mutable fixtures.\n* Supports lazy arguments generation including reading from file,\n  autogeneration etc.. (critical for large input datasets which does not fit\n  into RAM all together)\n* Output customization, separate tools for performance results comparison and\n  more.\n\n### measure time\n\n(Results below are obtained on 4 cores i7-2600 CPU @ 3.40GHz, 8Gb RAM\nfor sltbench dataset; see \"Metrics\" section for details; see\n[How to benchmark](doc/howtobenchmark.md) section for instructions)\n\n|                      |sltbench|googlebench|nonius|\n|----------------------|:------:|:---------:|:----:|\n| benchmark time, sec  | 87.45  |   416.44  |482.03|\n| average rel error, % |  0.4%  |    5.6%   |  2.6%|\n| maximum rel error, % |  2.9%  |   15.1%   | 60%  |\n\n4.7x times speedup might be useful for projects with big performance tests count. For the original project regression performance testing tooks about a week and testing time reduction matters.\n\nLess relative error means:\n* Less false speedups/slowdowns you get during testing. Economy of developer\n  time.\n* More precise performance changes may be obtained. If testing tool provides\n  15% relative error than 10% speedup couldn't be measured.\n\n### compilation time\n\nSltbench contains improvements on compilation time of benchmarks (not library!)\nfor the most popular api since 2.4.0. Benchmarks for functions without arguments\nand driver launch might be implemented in a more lightweight manner. See\n[Advanced user guide](doc/advanced_user_guide.md) for explanation how to reuse\nimprovements.\n\n(benchcompile metric results below are obtained on Intel(R) Core(TM) i5-8400 CPU\n@ 2.80GHz, clang-6.0.0, dataset=simple for optimized cases only)\n\n|                    | compilation time, sec |\n|--------------------|:---------------------:|\n| sltbench 2.4.0     | 2.5                   |\n| googlebench v1.5.1 | 16.5                  |\n\n\n# Motivating example\n\nThe following code registers `my_function` for timing. Just compile it (do not\nforget about includes) and the executable file will test `my_function` for\nperformance.\n\n```c++\nvoid my_function()\n{\n\tstd::vector\u003csize_t\u003e v(100000, 0);\n\tstd::sort(v.begin(), v.end());\n}\n\nSLTBENCH_FUNCTION(my_function);\n\nSLTBENCH_MAIN();\n```\n\n\n# Documentation\n\n* [Build instructions](doc/build.md)\n* [Examples](doc/examples.md)\n* [How to benchmark](doc/howtobenchmark.md) - follow this\n  instruction before you run any performance test\n* [How to analyse performance results](doc/howto_analyze_results.md)\n* [How to compare performance results](doc/compare_results.md)\n* [Advanced command line options](doc/commandlineoptions.md)\n* [Advanced user guide](doc/advanced_user_guide.md)\n* [Critical metrics for sltbench project and evaluation tools](tools/metrics/README.md)\n\n\n# License\n\nsltbench is released under the Apache 2.0 license\n","funding_links":[],"categories":["Performance Testing"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivafanas%2Fsltbench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fivafanas%2Fsltbench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivafanas%2Fsltbench/lists"}