{"id":15048012,"url":"https://github.com/mattkretz/virtest","last_synced_at":"2025-04-10T01:11:27.684Z","repository":{"id":56825706,"uuid":"85331149","full_name":"mattkretz/virtest","owner":"mattkretz","description":"header-only unit test framework","archived":false,"fork":false,"pushed_at":"2020-09-17T11:22:49.000Z","size":321,"stargazers_count":17,"open_issues_count":1,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-10T01:11:22.982Z","etag":null,"topics":["cpp","cpp11","test-framework","unit-testing"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mattkretz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-03-17T16:12:07.000Z","updated_at":"2025-01-04T18:45:19.000Z","dependencies_parsed_at":"2022-09-13T08:12:24.281Z","dependency_job_id":null,"html_url":"https://github.com/mattkretz/virtest","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattkretz%2Fvirtest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattkretz%2Fvirtest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattkretz%2Fvirtest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattkretz%2Fvirtest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mattkretz","download_url":"https://codeload.github.com/mattkretz/virtest/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248137886,"owners_count":21053775,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","cpp11","test-framework","unit-testing"],"created_at":"2024-09-24T21:06:56.358Z","updated_at":"2025-04-10T01:11:27.664Z","avatar_url":"https://github.com/mattkretz.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Vir's Unit Test Framework\n\n[![license](https://img.shields.io/github/license/mattkretz/virtest.svg)](https://github.com/mattkretz/virtest/blob/master/LICENSE)\n[![language](https://img.shields.io/badge/language-C%2B%2B11-blue.svg)](https://isocpp.org/)\n\n[![Build Status](https://travis-ci.org/mattkretz/virtest.svg)](https://travis-ci.org/mattkretz/virtest)\n[![Build status](https://ci.appveyor.com/api/projects/status/lxqk5tqs4og6dr3e?svg=true)](https://ci.appveyor.com/project/mattkretz/virtest)\n[![Codacy Badge](https://api.codacy.com/project/badge/Grade/77236083639c44a19c2d73609349f5a3)](https://www.codacy.com/manual/mattkretz/virtest?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=mattkretz/virtest\u0026amp;utm_campaign=Badge_Grade)\n\n## Why another test framework?\n\nThe test framework was developed inside the [Vc](https://github.com/VcDevel/Vc) repository.\nThe goal was to build a test framework supporting:\n\n* Minimal / no test registration or setup. Just write the test function and you're good.\n\n* Simple way to disable compilation of tests, without having to comment out sections of the source\n  file.\n\n* Simple instantiation of test code with types of a given list.\n\n* Support for fuzzy compares of floating point results with fine control over the ULP specification.\n\n* Assertion testing (i.e. verify that assertions fail on violated preconditions).\n\n* Simple but effective output (no XML, JSON, whatever; outputs a recognizable source location for more\n  effective test driven development)\n\nAll the test frameworks I looked at in 2009 (and 2010) did not even come close to supporting the\nabove requirements.\n\nSince I'm now very familiar with this test framework I want to use it in my other projects. The only\nsensible choice is to release the test framework on its own.\n\n## Usage\n\n### Creating an executable\nTo write a test executable all you need is to include the test header:\n```cpp\n#include \u003cvir/test.h\u003e\n```\n\nThis defines a main function, but at this point there are no tests, so it'll pass with the following\noutput:\n```\n Testing done. 0 tests passed. 0 tests failed. 0 tests skipped.\n```\n\n### Creating a test function\nSimple test functions are created with the `TEST` macro. Checks inside the test are done with\nmacros. The need for macros is due to the requirement to output the source location on failure. (The\nmacros `__FILE__` and `__LINE__` only yield the right value when expanded at the location of the\ntest.) You can use the following macros:\n\n* `COMPARE(value, reference)`\n  Compares `value` against `reference`, requiring the two to be equal. The comparison is done via\n  equality operator, if it is usable. If no equality operator is defined for the type a fallback to\n  memcmp is done. Note that this may yield incorrect failures if the types contain uninitialized\n  padding bits. Also, if the equality operator does not return a boolean, the implementation will\n  try to reduce the result to a boolean via calling `all_of(value == reference)`.\n\n* `COMPARE_TYPES(T1, T2)`\n  Test whether `T1` and `T2` are the same type (including value category). The \n  comparison is done via `std::is_same`. On failure this test macro prints the \n  `typeid` name wrapped inside a `vir::test::type\u003cT\u003e` template, to not lose \n  information about cv- and ref-qualifiers (`typeid` drops them).\n\n* `FUZZY_COMPARE(value, reference)`\n  Verifies that `value` is equal to `reference` within a pre-defined distance \n  in units of least precision (ulp). If the test fails it will print the \n  distance in ulp between `value` and `reference` as well as the maximum \n  allowed distance. Often this difference is not visible in the value because \n  the conversion of a double/float to a string needs to round the value to a \n  sensible length. The allowed distance can be modified by calling:\n  ```cpp\n  vir::test::setFuzzyness\u003cfloat\u003e(4);\n  vir::test::setFuzzyness\u003cdouble\u003e(7);\n  ```\n \n  * ulp: Unit of least precision is a unit that is derived from the least \n    significant bit in the mantissa of a floating-point value. Consider a \n    single-precision number (23 mantissa bits) with exponent *e*. Then 1 ulp is \n    *2ᵉ⁻²³*. Thus, *log₂(u)* signifies the number of incorrect mantissa bits \n    (with *u* the distance in ulp). If `value` and `reference` have a different \n    exponent the meaning of ulp depends on the variable you look at. The \n    `FUZZY_COMPARE` implementation always uses `reference` to determine the \n    magnitude of 1 ulp. Example: The value `1.f` is `0x3f800000` in binary. The \n    value `1.00000011920928955078125f` with binary representation `0x3f800001` \n    therefore has a distance of 1 ulp. A positive distance means the `value` is \n    larger than the `reference`. A negative distance means the `value` is \n    smaller than the `reference`.\n    * `FUZZY_COMPARE(1.00000011920928955078125f, 1.f)` will show a distance of \n      1 ulp\n\n    * `FUZZY_COMPARE(1.f, 1.00000011920928955078125f)` will show a distance of \n      -1 ulp\n      \n    The value `0.999999940395355224609375f` with binary representation \n    `0x3f7fffff` has a smaller exponent than `1.f`:\n    * `FUZZY_COMPARE(0.999999940395355224609375f, 1.f)` will show a distance of\n      -0.5 ulp\n\n    * `FUZZY_COMPARE(1.f, 0.999999940395355224609375f)` will show a distance of \n      1 ulp\n \n  * Comparing to 0: Distance to 0 is implemented as comparing to \n    `std::numeric_limits\u003cT\u003e::min()` instead and adding 1 to the resulting \n    distance.\n\n* `COMPARE_ABSOLUTE_ERROR(value, reference, error)`\n  As above, but allowing an absoluted difference between `value` and `reference`.\n  Verifies that the difference between `value` and `reference` is smaller than \n  `error`. If the test fails, the output will show the actual difference \n  between `value` and `reference`. If this value is positive `value` is too \n  large. If it is negative `value` is too small.\n\n* `COMPARE_RELATIVE_ERROR(value, reference, error)`\n  Verifies that the difference between `value` and `reference` is smaller than \n  `error * reference`. If the test fails, the output will show the actual \n  difference between `value` and `reference`. If this value is positive `value` \n  is too large. If it is negative `value` is too small. The following example \n  tests that `a` is no more than 1% different from `b`:\n  ```cpp\n  COMPARE_ABSOLUTE_ERROR(a, b, 0.01);\n  ```\n  If `reference` is set to 0, this macro compares the difference against `error *\n  \u003csmallest positive normalized value of reference type\u003e`.\n\n* `MEMCOMPARE(value, reference)`\n  Executes a memcmp over the storage bytes of `value` and `reference`. The \n  number of bytes compared is determined via `sizeof`.\n\n* `VERIFY(boolean)`\n  Passes if the argument converted to `bool` is `true`. Fails otherwise.\n\n* `FAIL()`\n  Immediately fails a test.\n\n* `vir::test::SKIP() \u003c\u003c \"details\"`\n  Ends the test, marking it as skipped but not failed.\n\n* `vir::test::ADD_PASS() \u003c\u003c \"details\"`\n  Counts and prints an additional passed test\n\n* `vir::test::expect_failure()`\n  The next `COMPARE`, `VERIFY`, etc. is expected to fail. The failure will \n  still end the test, but it will print `XFAIL` instead of `FAIL` and will not \n  count as a failed test in the summary.\n\n* `T vir::test::make_value_unknown(const T\u0026 x)`\n  The value returned from this function will be unknown to the compiler, \n  inhibiting constant propagation optimization passes. This can be important to \n  fully test whether an operation works correctly under all circumstances. Most \n  importantly, some unit tests may compile to nothing (identified as dead code, \n  i.e. code without side effects) if the compiler can infer the result from \n  constant inputs. In such cases it may be important to make test values \n  unknown to the compiler so that runtime behavior is actually tested.\n\n* `NOINLINE(\u003ctestable expression\u003e)`\n  When a test fails and you want to identify the exact instruction sequence \n  that lead to the failure, then wrapping the expression inside the `COMPARE` \n  or `VERIFY` macro with `NOINLINE` can help you. It places the expression \n  inside a return statement int a lambda which is executed in a function that \n  is guaranteed to not get inlined. Consequently, the instruction pointer \n  printed on failure takes you right after the `vir::test::noinline\u003c...\u003e` call \n  where the failing test was evaluated.\n\nExample:\n```cpp\nTEST(test_name) {\n  VERIFY(1 \u003e 0);\n  COMPARE(1, 1);\n\n  struct A { int x; };\n  COMPARE(A(), A());     // implicitly does memcmp\n  MEMCOMPARE(A(), A());  // explicitly does memcmp\n}\n```\n\n### Creating a test function instantiated from a typelist\n```cpp\nTEST_TYPES(T, test_name, (int, float, short)) {\n  COMPARE(T(), T(0));\n}\n```\n\n### Creating a test function that expects an exception\n```cpp\nTEST_CATCH(test_name, std::exception) {\n  throw std::exception();\n}\n```\n\nThis expects the code to throw an exception of the type specified as second \nmacro argument. If the code does not throw (or throws a different exception),\nthe test is considered failed.\n\n### Output additional information on failure\nEvery compare/verify macro acts as an output stream, usable for printing more \ninformation in the failure case. Alternatively, the `.on_failure(...)` function \ncan be used. Internally, `on_failure` still uses ostream operators to format \nthe output string.\n\nExample:\n```cpp\nTEST(test_name) {\n  int test = 3;\n  COMPARE(test, 2) \u003c\u003c \"more \" \u003c\u003c \"details\";\n  VERIFY(1 \u003e 0).on_failure(\"or \", \"like \", \"this\");\n}\n```\nPrints:\n```\n FAIL: ┍ at tests/testfile.cpp:5 (0x40451f):\n FAIL: │ test (3) == 2 (2) -\u003e false more details\n FAIL: ┕ test_name\n\n Testing done. 0 tests passed. 1 tests failed.\n```\n\n### Default output on failure\nNote in the output above it shows:\n\n1. The name of the test function that failed is at the end (`test_name`).\n\n2. The first line points to the source location of the `COMPARE` macro that \n   failed. In this case it's on line 5 of tests/testfile.cpp\n\n3. If you want to inspect the disassembly of the test, the failure was located \n   around 0x40451f.\n\n4. The `COMPARE` macro compared the expression `test`, which had value `3`, \n   against the expression `2`, which had value `2`. The result of `operator==` \n   is `false` (this can be useful information if `operator==` returns a \n   non-bool type).\n\n5. At the end of test executable, a summary of the test results is shown.\n\n### Testing assertions\nIf you have assertions using `\u003ccassert\u003e`'s `assert(cond)` macro in your code, \nyou can `#include \u003cvir/testassert.h\u003e` to replace the standard `assert` macro \nwith an implementation of the test framework. This enables two features:\n\n1. If an assertion is triggered from one of the tests, the test framework \n   recognizes it as a test failure and does not call `abort` like the standard \n   definition of the macro does.\n\n2. You can test whether pre-condition violations are recognized by the \n   assertions in your code. Wrap the code that violates the pre-condition with \n   `vir::test::expect_assert_failure([]() { violate_pre_condition(); })`. Now \n   the test fails if the assertion holds.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmattkretz%2Fvirtest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmattkretz%2Fvirtest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmattkretz%2Fvirtest/lists"}