Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sunxfancy/zeroerr
A powerful C++ unit testing/assert/log utils
https://github.com/sunxfancy/zeroerr
assert assertion benchmarking c-plus-plus-11 cpp cpp11 dbg fuzzing header-only logger logging test-framework unit-test unit-testing unit-testing-framework
Last synced: about 20 hours ago
JSON representation
A powerful C++ unit testing/assert/log utils
- Host: GitHub
- URL: https://github.com/sunxfancy/zeroerr
- Owner: sunxfancy
- License: mit
- Created: 2022-06-06T21:26:50.000Z (over 2 years ago)
- Default Branch: master
- Last Pushed: 2024-12-13T21:25:36.000Z (10 days ago)
- Last Synced: 2024-12-13T22:26:02.428Z (10 days ago)
- Topics: assert, assertion, benchmarking, c-plus-plus-11, cpp, cpp11, dbg, fuzzing, header-only, logger, logging, test-framework, unit-test, unit-testing, unit-testing-framework
- Language: C++
- Homepage: https://sunxfancy.github.io/zeroerr/en/html/
- Size: 8.64 MB
- Stars: 24
- Watchers: 3
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: Readme.md
- Contributing: Contributing.en.md
- License: LICENSE
Awesome Lists containing this project
README
# ZeroErr
[](https://en.wikipedia.org/wiki/C%2B%2B#Standardization) [](https://raw.githubusercontent.com/sunxfancy/zeroerr/master/zeroerr.hpp) [](./Readme.zh.md)
[](https://replit.com/@sunxfancy/ZeroErr-Demo#main.cpp)
Hope you get 0 errors and 0 warnings everyday!
ZeroErr is a smart assertion library, a lightweight unit testing framework and a structure logging framework. It integrates those features and provided an unite and clear interface for separated usage or combined usage.
[English Documentation](https://sunxfancy.github.io/zeroerr/en/html/) | [项目文档](https://sunxfancy.github.io/zeroerr/zh/html/)
Note: The project is currently in the experimental stage, and the API may change significantly. It is not recommended to use it in a production environment.
## Why we need another unit testing framework
The current popular unit testing frameworks, e.g. Catch2, doctest, Boost.Test and cpputest are mature and well-established which covers common cases during development. The logger libraries like glog and spdlog are also easy to use. However, there are still some issues:
### 1. Generic Printing
Most unit testing frameworks and logger libraries can not provide a generic printing for user customized types. Especially, when using containers, structures and pointers (including smart pointers), user have to manually write code to generate the log message or print those information during unit testing failed cases.
This library `zeroerr` gives you an ability to print generically for all types:
```c++
TEST_CASE("Try logging") {
std::vector data = {1, 2, 3};
LOG_IF(1 == 1, "data = {data}", data);
}
```
Similar to other C++ unit testing frameworks, `zeroerr` will convert this piece of code into a function and register it to automatically run once you link the main function and the library. Here, we can log the data in `vector` template directly without writing any code.
For the custom struct type with override `std::ostream& operator<<(std::ostream&, Type)` stream output, you can use it not only for this type but also all contains using this type, including multiple recursive contains:
```c++
struct Node {
std::string name;
int id;
};
std::ostream& operator<<(std::ostream& out, Node n) {
out << n.id << '.' << n.name;
return out;
}
TEST_CASE("Try logging with custom type") {
std::map> data = {
{"user1", {{"a",1}, {"b",2}}}, {"user2", {{"c",3}, {"d",4}}}
};
LOG("data = {data}", data);
}
```
Of cause, in many cases, some third-party libraries may not use `<<` operators. For those cases, we can write own rules to create a generic way for printing. For example, LLVM `llvm::Function*` type can not be streamed into std::ostream, we can write code to handle it. However, it will be more simple if we can write a rule for all the sub-classes of `llvm::Value` and `llvm::Type` since we can call the `print` method to print the output. Here we use a `dbg` marco defined in `zeroerr` to quickly print any type. This is very similar to the `dbg` marco in rust.
```c++
namespace zeroerr { // must defined in namespace zeroerr
template
typename std::enable_if<
std::is_base_of::value || std::is_base_of::value, void>::type
PrinterExt(Printer& P, T* s, unsigned level, const char* lb, rank<2>) {
if (s == nullptr) {
P.os << P.tab(level) << "nullptr" << lb;
} else {
llvm::raw_os_ostream os(P.os);
s->print(os);
}
}
}
TEST_CASE("customize printing of LLVM pointers") {
llvm::LLVMContext context;
std::vector args = {llvm::Type::getInt32Ty(context)};
llvm::Module* module = new llvm::Module("test_module", context);
auto* f =
llvm::Function::Create(llvm::FunctionType::get(llvm::Type::getVoidTy(context), args, false),
llvm::GlobalValue::ExternalLinkage, "test", module);
dbg(dbg(f)->getType());
}
```
This functin `PrintExt` will match all the class who's base class is `Value` and `Type`. Then, it will create a stream ``llvm::raw_os_ostream` for output.
### 2. Combined usage of assert, log and unit testing
If you use one logging framework, an unit testing framework and an assertion library, it's not a easy work to combine them together. There is a lot of benefits to use assertion, logging and unit testing together. In `zeroerr`, if an assertion is failed, the logger will receive an event and stored the event in your log file. If you are using an assertion in unit testing, the assertion failure, logged fatal events can be recorded and reported.
```c++
int fib(int n) {
REQUIRE(n >= 0, "n must be non-negative");
REQUIRE(n < 20, "n must be less than 20");
if (n <= 2) {
return 1;
}
return fib(n - 1) + fib(n - 2);
}
TEST_CASE("fib function test") {
CHECK(fib(0) == 0);
CHECK(fib(1) == 1);
CHECK(fib(2) == 1);
CHECK(fib(3) == 2);
CHECK(fib(4) == 3);
CHECK(fib(5) == 5);
CHECK(fib(20) == 6765);
}
```
For the logging system, the unit testing can access the log data to ensure that the function has executed the expected logic and results.
```c++
118 static void function() {
119 int k = system_call();
120 LOG_IF(k != 0, "System call failed, error code = {k}", k);
121 }
...
TEST_CASE("access log in Test case") {
zeroerr::suspendLog();
function();
CHECK(LOG_GET(function, 120, k, int) == ERROR_CODE);
zeroerr::resumeLog();
}
```
In order to access the log, we need to pause the log system first, to avoid the data being output to the file, then call the function, access the data in the log through the `LOG_GET` macro, and finally resume the log system. (Currently experimental, only the first call of each log point can be accessed)
Further more, the unit testing can check the logged result if it matches the previous running result (a golden file) to avoid writing any code in the test case.
```c++
TEST_CASE("match ostream") {
// match output can be done in the following workflow
// 1. user mark the test case which are comparing output use 'ZEROERR_HAVE_SAME_OUTPUT'
// 2. If the output is not exist, the result will be store to the disk.
// 3. If the output is exist, compare with it and report error if output is not match.
std::cerr << "a = 100" << std::endl;
ZEROERR_HAVE_SAME_OUTPUT;
}
```
Once you set `ZEROERR_HAVE_SAME_OUTPUT` marco, the system will check the output stream and save the first run result into a file. Then, the next run will compare the result to see if it the same. (Currently experimental)
## 3. Fuzzing Support
Most Unit Testing frameworks do not support fuzzing. However, it's a powerful feature to automatically detect faults in the software and can greatly reduce the work to write test cases.
Different than other fuzzing framework, `zeroerr` can also support logging and assertion in the code, so the fuzzing result not only contains corpus but also with the logging and assertion information.
Here is an example of using `zeroerr` to do structured fuzzing:
```c++
FUZZ_TEST_CASE("fuzz_test") {
LOG("Run fuzz_test");
FUZZ_FUNC([=](int k, std::string num) {
int t = atoi(num.c_str());
LOG("k: {k}, num:{num}, t: {t}", k, num, t);
REQUIRE(k == t);
})
.WithDomains(InRange(0, 10), Arbitrary())
.WithSeeds({{5, "Foo"}, {10, "Bar"}})
.Run(10);
}
```
Inspired by [fuzztest](https://github.com/google/fuzztest), Domain is a concept to specify the input data range (or patterns) for the target function. Here, we use `InRange` to specify the range of `k` is 0 to 10, and `Arbitrary` to specify the data of `num` can be any random string. Then, we use `WithSeeds` to specify the initial seeds for the fuzzing.
The macro `FUZZ_TEST_CASE` will generate a test case which can connect with `libFuzzer` to run the fuzzing. Finally, we use `Run(10)` to call `libFuzzer` to run the target for 10 times.
To build the test case with fuzzing, you need to use `clang++` to compile the code and with `-fsanitize=fuzzer-no-link` and link the `-lclang_rt.fuzzer_no_main-x86_64` which is a version of libFuzzer without main function. You can find this runtime library by calling `clang++ -print-runtime-dir`. Here is the complete command to build the test case with fuzzing support:
```bash
clang++ -std=c++11 -fsanitize=fuzzer-no-link -L=`clang++ -print-runtime-dir` -lclang_rt.fuzzer_no_main-x86_64 -o test_fuzz test_fuzz.cpp
```
## Other Good Features
Here are a list of features we provided:
1. Partially include
You can only include what you need. If you need only assertion but no unit testing, no problem.
2. Optional thread safety
You can choose to build with/without thread safety.
3. Fastest log
Multiple level of log writing policies. You can choose to only write to disk with the most important events.
4. Customized print / log / assert printing format
You can customize your printing format for everything. There is a templated callback function for the printing.
5. Quickly debug something
You can use dbg macro to quickly see the output, it can be applied to any expression.
6. Colorful output
You can have default colorful output to terminal and no color for file output
7. Print struct/stl/pointers without any extra code
8. Doctest like assertion and unit test feature
You can use your unit test as a documentation of function behavior. The output of unittest can be a documented report.
9. Lazy logging for assertion
After assertion failed, the logging result will print automatically even if you didn't redirect to your error stream
10. Logging Category
Logging information can have customized category and only display one category based on your assertion or configuration
11. Logging for Unit Testing
You can use a correct logging result as your unit testing golden file. So you just need to manually verify your log once and save it. The unit testing framework will use the golden file to verify your unit testing result.
12. Structured Logging
We can support output structured information directly into plain text or lisp format (json, logfmt, or other custom format should be the next step to support)
13. Automatic Tracing with logging
While logging at the end, we can record the time consuming for this function.
## Header-only libraries
* dbg
* print (without use extern functions)
* assert
* color (if always enabled)
## The logo generation
Thanks to the `tiv` tool:
https://github.com/stefanhaustein/TerminalImageViewer