Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/bxparks/aunit

Unit testing framework for Arduino platforms inspired by ArduinoUnit and Google Test. Used with EpoxyDuino for continuous builds.
https://github.com/bxparks/aunit
arduino arduino-library esp32 esp8266 teensyduino unit-testing unit-testing-framework
Last synced: about 1 month ago
JSON representation
Unit testing framework for Arduino platforms inspired by ArduinoUnit and Google Test. Used with EpoxyDuino for continuous builds.
Host: GitHub
URL: https://github.com/bxparks/aunit
Owner: bxparks
License: mit
Created: 2018-03-13T04:36:09.000Z (almost 7 years ago)
Default Branch: develop
Last Pushed: 2023-10-31T21:46:31.000Z (about 1 year ago)
Last Synced: 2024-10-13T21:05:47.043Z (2 months ago)
Topics: arduino, arduino-library, esp32, esp8266, teensyduino, unit-testing, unit-testing-framework
Language: C++
Homepage:
Size: 7.02 MB
Stars: 175
Watchers: 9
Forks: 16
Open Issues: 3
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project

README

        # AUnit

[![AUnit Tests](https://github.com/bxparks/AUnit/actions/workflows/aunit_tests.yml/badge.svg)](https://github.com/bxparks/AUnit/actions/workflows/aunit_tests.yml)

A unit testing framework for Arduino platforms inspired by by

[ArduinoUnit](https://github.com/mmurdoch/arduinounit) and [Google

Test](https://github.com/google/googletest/). AUnit is almost a drop-in

replacement of ArduinoUnit (v2.2) with some advantages. AUnit supports timeouts

and test fixtures. It sometimes consumes 50% less flash memory on the AVR

platform, and it has been tested to work on the AVR, SAMD21, STM32, ESP8266,

ESP32 and Teensy platforms. The assertion error messages were updated in v1.7 to

provide seamless integration with Unix tools like `vim`.

Originally, the AUnit tests were designed to run on the embedded controller

itself which allows detection of architecture-specific problems. But the

uploading, flashing, and execution process is often slow and flaky, causing the

iteration cycle to take too much time. It is often more effective to execute the

AUnit tests natively on a host machine (running Linux, MacOS, or FreeBSD) using

the [EpoxyDuino](https://github.com/bxparks/EpoxyDuino) companion project. Once

the unit tests are running on the Linux or MacOS host machine, they can be

incorporated into a continuous integration system like

[Jenkins](https://www.jenkins.io/) system or a cloud-based system like [GitHub

Actions](https://github.com/features/actions).

**Version**: 1.7.1 (2023-06-15)

**Changelog**: [CHANGELOG.md](CHANGELOG.md)

## Table of Contents

* [Summary](#Summary)

    * [ArduinoUnit Compatible Features](#ArduinoUnitCompatible)

    * [Missing Features](#MissingFeatures)

    * [Added Features](#AddedFeatures)

* [Installation](#Installation)

    * [Source Code](#SourceCode)

* [Documentation](#Documentation)

    * [Examples](#Examples)

* [Usage](#Usage)

    * [Header and Namespace](#HeaderAndNamespace)

    * [Verbose Mode](#VerboseMode)

    * [Defining the Tests](#DefiningTests)

    * [Generated Class and Instance Names](#GeneratedClass)

    * [Binary Assertions](#BinaryAssertions)

        * [Assertion Message Format](#AssertionMessageFormat)

        * [Supported Parameter Types](#SupportedParameterTypes)

        * [Parameter Types Must Match](#ParameterTypesMustMatch)

        * [Pointer Comparisons](#PointerComparisons)

        * [Case Insensitive String Comparisons](#CaseInsensitiveStrings)

    * [Approximate Comparisons](#ApproximateComparisons)

    * [Boolean Assertions](#BooleanAssertions)

    * [Test Fixtures](#TestFixtures)

    * [Early Return and Delayed Assertions](#EarlyReturnDelayedAssertions)

    * [Meta Assertions](#MetaAssertions)

    * [Unconditional Termination](#UnconditionalTermination)

    * [Overridable Methods](#OverridableMethods)

    * [Running the Tests](#RunningTests)

    * [Filtering Test Cases](#FilteringTestCases)

    * [Output Printer](#OutputPrinter)

    * [Controlling Verbosity](#ControllingVerbosity)

    * [Line Number Mismatch](#LineNumberMismatch)

    * [Test Framework Messages](#TestFrameworkMessages)

        * [Assertion Message](#AssertionMessage)

        * [Verbose Mode Message](#VerboseModeMessage)

        * [Test Case Summary](#TestCaseSummary)

        * [Test Runner Summary](#TestRunnerSummary)

    * [Test Timeout](#TestTimeout)

* [GoogleTest Adapter](#GoogleTestAdapter)

* [Command Line Tools](#CommandLineTools)

    * [AUniter](#AUniter)

    * [EpoxyDuino](#EpoxyDuino)

    * [Command Line Flags and Arguments](#CommandLineFlagsAndArguments)

* [Continuous Integration](#ContinuousIntegration)

    * [Arduino IDE/CLI + Cloud](#IdePlusCloud)

    * [Arduino IDE/CLI + Jenkins](#IdePlusJenkins)

    * [EpoxyDuino + Jenkins](#EpoxyDuinoPlusJenkins)

    * [EpoxyDuino + Cloud (Recommended)](#EpoxyDuinoPlusCloud)

* [Tips](#Tips)

    * [Debugging Assertions in Fixtures](#DebuggingFixtures)

    * [Class Hierarchy](#ClassHierarchy)

    * [Testing Private Helper Methods](#PrivateHelperMethods)

* [Benchmarks](#Benchmarks)

    * [Memory Benchmark](#MemoryBenchmark)

    * [Compared with ArduinoUnit](#CompareArduinoUnit)

* [System Requirements](#SystemRequirements)

    * [Hardware](#Hardware)

    * [Toolchains](#Toolchains)

    * [Operating System](#OperatingSystem)

* [License](#License)

* [Feedback and Support](#FeedbackAndSupport)

* [Authors](#Authors)



## Summary

AUnit was created to solve 3 problems with ArduinoUnit 2.2:

* ArduinoUnit consumes too much flash memory on an AVR platform (e.g.

  Arduino UNO, Nano) as explained in

  [ArduinoUnit#70](https://github.com/mmurdoch/arduinounit/issues/70).

* ArduinoUnit does not compile on the ESP8266 platform (see

  [ArduinoUnit#68](https://github.com/mmurdoch/arduinounit/issues/68),

  [ArduinoUnit#55](https://github.com/mmurdoch/arduinounit/issues/55),

  [ArduinoUnit#54](https://github.com/mmurdoch/arduinounit/issues/54)).

* ArduinoUnit does not provide an easy way to create tests using fixtures,

  equivalent to the `TEST_F()` macro in Google Test.

In contrast:

* AUnit consumes as much as 65% *less* flash memory than ArduinoUnit 2.2 on the

  AVR platform. On Teensy-ARM, the savings can be as much as 30%.

* AUnit has been tested on AVR, Teensy-ARM and ESP8266.

* AUnit implements the `testF()` and `testingF()` macros to use fixtures.

The [AUniter](https://github.com/bxparks/AUniter) command line tools can compile

the unit tests, upload them to Arduino boards attached to the serial ports of

the local machine, and validate the output of the AUnit test runner. In

addition, the AUniter script can be integrated into a

[Jenkins](https://jenkins.io) continuous integration service running on the

local machine, and the unit tests can be monitored automatically.

Unit tests written using AUnit can often be compiled and executed natively on

Linux or MacOS using the

[EpoxyDuino](https://github.com/bxparks/EpoxyDuino) library. The output on

the `Serial` object is redirected to the `stdout` of the Unix host. This

provides another avenue for implementing continuous builds or integration.



### ArduinoUnit Compatible Features

For basic unit tests written using ArduinoUnit 2.2, only two changes are

required to convert to AUnit:

* `#include ` -> `#include `

* `Test::run()` -> `aunit::TestRunner::run()`

Most of the core macros are compatible between ArduinoUnit and AUnit:

* `test()`

* `testing()`

* `assertXxx()`

* Meta Assertions

    * `checkTestXxx()`

    * `assertTestXxx()`

* `externTest()`

* `externTesting()`

AUnit also supports exclude and include filters:

* `TestRunner::exclude()`

* `TestRunner::include()`

Filters can be accessed through

[Command Line Flags](##CommandLineFlagsAndArguments) on desktop machines using

EpoxyDuino

The various assertion and test status messages can be enabled or disabled using

the `Verbosity` flags on per test basis:

* `enableVerbosity()`

* `disableVerbosity()`



### Missing Features

Here are the features which have *not* been ported over from ArduinoUnit 2.2:

* ArduinoUnit supports multiple `*` wildcards in its `exclude()` and `include()`

  methods. AUnit supports only a single `*` wildcard and it must occur at the

  end if present.



### Added Features

Here are the features in AUnit which are not available in ArduinoUnit 2.2:

* Configurable timeout parameter to prevent `testing()` test cases from

  running forever:

    * `TestRunner::setTimeout(seconds)`

    * `Test::expire()`

    * `assertTestExpire()`

    * `assertTestNotExpire()`

    * `checkTestExpire()`

    * `checkTestNotExpire()`

* Case-insensitive string comparisons:

    * `assertStringCaseEqual()`

    * `assertStringCaseNotEqual()`

* Approximate comparisons:

    * `assertNear()`

    * `asssertNotNear()`

* Supports 64-bit integer

    * `assertXxx()` support both `long long` and `unsigned long long`

* `test()` and `testing()` macros support both 1 and 2 arguments

    * `test(testName)` and `test(suiteName, testName)`

    * `testing(testName)` and `testing(suiteName, testName)`

* Test fixtures using the "F" variations of existing macros:

    * `testF()`

    * `testingF()`

    * `assertTestXxxF()`

    * `checkTestXxxF()`

    * `externTestF()`

    * `externTestingF()`

* Unconditional termination:

    * `passTestNow()`

    * `failTestNow()`

    * `skipTestNow()`

    * `expireTestNow()`

* `teardown()` method which mirrors the `setup()` method:

    * `teardown()`

* Test filters support 2-arguments, matching `testF()` and `testingF()`:

    * `TestRunner::include(testClass, name)`

    * `TestRunner::exclude(testClass, name)`

* No dynamic allocation of heap memory:

    * The library does not create a `String` object.

    * It reads from a `String` if the calling code provides it.

* Terse and verbose modes:

    * `#include ` - terse messages use less flash memory

    * `#include ` - verbose messages use more flash memory

Every feature of AUnit is unit tested using AUnit itself.



## Installation

The latest stable release is available in the Arduino IDE Library Manager.

Search for "unit test" or "AUnit", select "AUnit", then click the "Install"

button.

The development version can be installed by cloning the

[GitHub repository](https://github.com/bxparks/AUnit), checking out the

`develop` branch, then manually copying over the contents to the `./libraries`

directory used by the Arduino IDE. (The result is a directory named

`./libraries/AUnit`.) See the Preferences menu for the location of the Arduino

sketch directory. The `master` branch contains the stable release.

Using either installation method, you may need to restart the Arduino IDE to

pick up the new library.



### Source Code

The source files are organized as follows:

* `src/AUnit.h` - main header file

* `src/AUnitVerbose.h` - verbose version of main header file

* `src/aunit/` - all implementation files

* `tests/` - unit tests written using AUnit itself

* `examples/` - example sketches



## Documentation

* [README.md](README.md) - this file

* [Doxygen docs](https://bxparks.github.io/AUnit/html) published on GitHub

  Pages can help navigate the classes and header files



### Examples

The `examples/` directory has a number of examples:

* Basic

    * [basic](examples/basic)

        * using the `test()` macro

    * [fixture](examples/fixture)

        * how to use the `testF()` macro with test fixtures

* Intermediate

    * [filter](examples/filter)

        * how to filter tests using `TestRunner::include()` and

          `TestRunner::exclude()`

    * [meta_asserts](examples/meta_asserts)

        * how to use `assertTestXxx()` and `checkTestXxx()`

* Advanced

    * [advanced](examples/advanced/)

        * how to subclass `Test` and `TestOnce` manually

    * [continuous](examples/continuous)

        * using the `testing()` macro

* Benchmarks

    * These are internal benchmark programs, not meant as examples (they are in

      the `examples/` directory because of the Arduino IDE).

    * [MemoryBenchmark](examples/MemoryBenchmark)

        * Determines the flash and static memory consumption of AUnit for

          various microcontroller

In the `tests/` directory, there are unit tests to test the AUnit framework

itself:

* [AUnitTest](tests/AUnitTest)

    * the unit test for core `AUnit` functions

* [AUnitMoreTest](tests/AUnitMoreTest)

    * more tests

* [AUnitMetaTest](tests/AUnitMetaTest)

    * the unit test for meta assertions and `extern*()` macros

* [CompareTest](tests/CompareTest)

    * tests for the low-level compare functions

* [FailingTest](tests/FailingTest)

    * tests that are expected to fail

* [FilterTest](tests/FilterTest)

    * manual tests for `include()` and `exclude()` filters

* [Print64Test](tests/Print64Test)

    * manual tests for `include()` and `exclude()` filters

* [SetupAndTeardownTest](tests/SetupAndTeardownTest)

    * tests to verify that `setup()` and `teardown()` are called properly by the

      finite state machine

* [tests/Makefile](tests/Makefile)

    * Runs the tests on a Linux or Mac machine using EpoxyDuino

Perhaps the best way to see AUnit in action is through real life examples. All

my libraries use AUnit for testing and for continuous integration through

EpoxyDuino. Here are some of my libraries:

* [AceButton](https://github.com/bxparks/AceButton)

    * My first Arduino library, which originally used ArduinoUnit 2.2.

    * I kept many of the original ArduinoUnit tests for backwards compatibility

      testing. But over time, I started to use more AUnit features so I'm not

      sure if they work with ArduinoUnit anymore.

* [AceCRC](https://github.com/bxparks/AceCRC)

* [AceCommon](https://github.com/bxparks/AceCommon)

* [AceRoutine](https://github.com/bxparks/AceRoutine)

* [AceSegment](https://github.com/bxparks/AceSegment)

* [AceSorting](https://github.com/bxparks/AceSorting)

* [AceTime](https://github.com/bxparks/AceTime)



## Usage

In this section, information about differences between AUnit and ArduinoUnit

will appear in a note marked by ***ArduinoUnit Compatibility***.



### Header and Namespace

To prevent name clashes with other libraries and code, all classes in the AUnit

library are defined in the `aunit` namespace. The user will mostly interact with

the `TestRunner` class. It can be referenced with an explicit namespace

qualifier (i.e. `aunit::TestRunner`), or we can use a `using` directive like

this:

```C++

#include 

using aunit::TestRunner;

```

or we can import the entire `aunit` namespace:

```C++

#include 

using namespace aunit;

```

Similar to ArduinoUnit, many of the "functions" in this framework (e.g.

`test()`, `testing()`, `assertXxx()`) are defined as `#define` macros which live

in the global namespace, so it is usually not necessary to import the entire

`aunit` namespace.



### Verbose Mode

By default, AUnit generates terse assertion messages by leaving out

the string arguments of the various `assertXxx()` macros. If you would like

to get the same verbose output as ArduinoUnit, use the following header

instead:

```C++

#include 

```

The flash memory consumption on an 8-bit AVR may go up by 20-25% for medium to

large tests. On Teensy ARM, ESP8266 or ESP32, the increased memory size probably

does not matter too much because these microcontrollers have far more flash and

static memory.



### Defining the Tests

The usage of **AUnit** is basically identical to **ArduinoUnit**. The following

macros are used to create a test:

* `test(name) {...}` - creates a subclass of `TestOnce`

* `testing(name) {...}` - creates a subclass of `TestAgain`

* `test(suiteName, name) {...}` - creates a subclass of `TestOnce`

* `testing(suiteName, name) {...}` - creates a subclass of `TestAgain`

* `testF(classname, name) {...}` - creates a subclass of `classname`

* `testingF(classname, name) {...}` - creates a subclass of `classname`

The code in `{ }` following these macros becomes the body of a method in a

subclass derived from the base class indicated above. The `test()` and `testF()`

macros place the code body into the `TestOnce::once()` method. The `testing()`

and `testingF()` macros place the code body into the `TestAgain::again()`

method.

The `test()` and `testing()` macros support 1 or 2 arguments. The one-argument

version is inherited from ArduinoUnit. The two-argument version is

analogous to the `TEST()` macro in GoogleTest, where the `suiteName` can

be used to organize multiple tests into a collection of similar tests. The

grouping is purely in the naming scheme of the generated code, there is no

functional relationship between these tests.

During static initialization, the constructor of the object adds itself to an

internal list. The root of that list is given by `Test::getRoot()`. The

`TestRunner::run()` method traverses the linked list, executing each test case

until it passes, fails or is skipped.

Here is a rough outline of an AUnit unit test sketch:

```C++

#line 2 "AUnitTest.ino"

#include 

using namespace aunit;

test(example) {

  ...

  assertXxx(...)

  ...

}

test(ExampleTest, example) {

  ...

}

testing(looping) {

  ...code...

  if (...) {

    pass();

  } else if (...) {

    failTestNow();

  } else {

    skipTestNow();

  }

}

testing(LoopingTest, looping) {

  ...

}

class CustomTestOnce: public TestOnce {

  protected:

    // optional

    void setup() override {

      TestOnce::setup();

      ...setup code...

    }

    // optional

    void teardown() override {

      ...teardown code...

      TestOnce::teardown();

    }

    void assertBigStuff() {

      ...higher level assertions...

    }

};

testF(CustomTestOnce, example_test) {

  ...

  assertBigStuff();

  ...

}

class CustomTestAgain: public TestAgain {

  protected:

    // optional

    void setup() override {

      TestAgain::setup();

      ...setup code...

    }

    // optional

    void teardown() override {

      ...teardown code...

      TestAgain::teardown();

    }

    void assertBigStuff() {

      ...various assertions...

    }

};

testingF(CustomTestAgain, example_test) {

  ...

  assertBigStuff();

  ...

}

void setup() {

  Serial.begin(115200);

  while (! Serial); // Wait until Serial is ready - Leonardo/Micro

  TestRunner::exclude("*");

  TestRunner::include("looping*");

  TestRunner::include("CustomTestAgain", "example*");

}

void loop() {

  TestRunner::run();

}

```

***ArduinoUnit Compatibility***: _The basic structure of the unit test is

identical to ArduinoUnit. AUnit adds the `testF()` and `testingF`() macros,

and the two-argument versions of `test()` and `testing()` which

are not available in ArduinoUnit. The `Test` class in ArduinoUnit has been

replaced with the `TestAgain` class in AUnit._



### Generated Class and Instance Names

The arguments to the various `test*()` macros are used to generate the name for

the subclasses of `TestOnce` or `TestAgain`, and generate the names of the

instances of those classes. For reference, here are the rules:

* `test(name)`

    * class: `"test_"` + name

    * instance: `"test_"` + name + `"_instance"`

* `testing(name)`

    * class: `"test_"` + name

    * instance: `"test_"` + name + `"_instance"`

* `test(suiteName, name)`

    * class: `suiteName` + `"_"` + name

    * instance: `suiteName` + `"_"` + name + `"_instance"`

* `testing(suiteName, name)`

    * class: `suiteName` + `"_"` + name

    * instance: `suiteName` + `"_"` + name + `"_instance"`

* `testF(className, name)`

    * class: `className` + `"_"` + name

    * instance: `className` + `"_"` + name + `"_instance"`

* `testingF(className, name)`

    * class: `className` + `"_"` + name

    * instance: `className` + `"_"` + name + `"_instance"`

The instance name is available within the test code using the `Test::getName()`

method.



### Binary Assertions

Inside the `test()` and `testing()` macros, the following assertions

are available. These are essentially identical to ArduinoUnit:

* `assertEqual(a, b)`

* `assertNotEqual(a, b)`

* `assertLess(a, b)`

* `assertMore(a, b)`

* `assertLessOrEqual(a, b)`

* `assertMoreOrEqual(a, b)`



#### Assertion Message Format

When the assertion passes, nothing is printed by default. This can be controlled

by the `TestRunner::setVerbosity()` method. See [Controlling

Verbosity](#ControllingVerbosity).

When the assertion fails, an error message of the following format is printed:

```

SampleTest.ino:10: Assertion failed: (2) == (1)

```

The format of the assertion failure messages was changed in v1.7 to the

following:

```

{fileName}:{lineNumber}: Assertion failed: {expression}

```

This format is a widely used in many other programs, for example, the C compiler

`gcc`, the C++ compiler `g++`, the Python 3 interpreter `python3`, `grep`, and

the GNU Make program `make`. In particular, the

[quickfix](https://vimhelp.org/quickfix.txt.html) feature in the `vim` text

editor can parse this error format and jump directly to the `fileName` and

`lineNumber` indicated by the error message. See the instructions in

[EpoxyDuino](https://github.com/bxparks/EpoxyDuino) to see how to run unit tests

on a Linux or MacOS machine inside the `vim` editor so that the editor jumps

directly to the files and line numbers where the assertion failure occurred.



#### Supported Parameter Types

The 6 core assert macros (`assertEqual()`, `assertNotEqual()`, `assertLess()`,

`assertMore()`, `assertLessOrEqual()`, `assertMoreOrEqual()`) support the

following 18 combinations for their parameter types:

* `(bool, bool)`

* `(char, char)`

* `(int, int)`

* `(unsigned int, unsigned int)`

* `(long, long)`

* `(unsigned long, unsigned long)`

* `(long long, long long)`

* `(unsigned long long, unsigned long long)`

* `(double, double)`

* `(const char *, const char *)`

* `(const char *, const String&)`

* `(const char *, const __FlashStringHelper*)`

* `(const String&, const char*)`

* `(const String&, const String&)`

* `(const String&, const __FlashStringHelper*)`

* `(const __FlashStringHelper*, const char*)`

* `(const __FlashStringHelper*, const String&)`

* `(const __FlashStringHelper*, const __FlashStringHelper*)`

The `assertEqual()` and `assertNotEqual()` support arbitrary pointer types

through implicit casts to `const void*`:

* `(const void*, const void*)` (since v1.4)

All 9 combinations of the 3 string types (`char*`, `String`, and

`__FlashStringHelper*`) are supported.

These macros perform deep comparisons for string types instead of just comparing

their pointer values. This is different than the `ASSERT_EQ()` and `ASSERT_NE()`

macros in Google Test which perform only pointer comparisons. In other words,

`assertEqual()` with string types is equivalent to `ASSERT_STREQ()` in Google

Test.

Also for string types, these macros support `nullptr` (unlike the underlying

`strcmp()` function from the C-library). The `nullptr` string is defined to be

"smaller" than any non-null string, including the empty string. Two `nullptr`

strings are considered to be equal however.

Additionally, the usual C++ implicit type conversion and function overloading

matching algorithms apply to support additional argument types.

For example, the following type conversions will occur:

* `signed char` -> `int`

* `unsigned char` -> `int`

* `short` -> `int`

* `unsigned short` -> `int` or `unsigned int` (depending on `sizeof(int)`)

* `char*` -> `const char*`.

* `char[N]` -> `const char*`

* `float` -> `double`

* pointer types -> `const void*`

Note that `char`, `signed char`, and `unsigned char` are 3 distinct types in

C++, so a `(char, char)` will match exactly to one of the `assertXxx()`

methods.

***ArduinoUnit Compatibility***:

_The names of the macros are identical. However, the

type inference logic of two `(a, b)` arguments in the `assertXxx(a, b)` is

slightly different. ArduinoUnit allows the two parameters to be slightly

different types, at the expense of a compiler warning. In AUnit, the

warning becomes a compiler error. See the "Parameters Must Match Types" section

below._



#### Parameter Types Must Match

In ArduinoUnit, the `assertXxx()` macros could be slightly different types, for

example:

```C++

unsigned int uintValue = 5;

assertEqual(5, uintValue);

```

If the compiler warnings are enabled in the Preferences box of the

IDE, a warning from the compiler is printed:

```

../ArduinoUnit/src/ArduinoUnitUtility/Compare.h:17:28: warning:

    comparison between signed and unsigned integer expressions [-Wsign-compare]

    return (!(a, unsigned int&)' is ambiguous

    if (!aunit::assertion(__FILE__,__LINE__,(arg1),opName,op,(arg2)))\

...

```

The compiler cannot find an appropriate overloaded version of `assertEqual()`.


The solution is to make the parameters the same type:

```C++

assertEqual(5U, uintValue);

```

On the AVR platform, both a (short) and (int) are 16-bit types, so the following

will produce a compiler error:

```C++

unsigned short ushortValue = 5;

assertEqual(5U, ushortValue);

```

But on Teensy-ARM, ESP8266, and ESP32, a 16-bit (short) can be promoted to a

32-bit (int) without loss of precision, so the above will compile just fine. For

portability, the following should be used on all platforms:

```C++

unsigned short ushortValue = 5;

assertEqual((unsigned short) 5, ushortValue);

```

The integer type promotion rules and function overload matching rules can be

difficult to remember (and sometimes difficult to understand). The best way to

avoid these compiler errors is to make sure that the assertion parameter types

are identical, potentially using explicit casting.



#### Pointer Comparisons

Version 1.4 adds pointer comparison to `assertEqual()` and `assertNotEqual()`.

Arbritary pointers are implicitly cast to a `const void*` and compared to

each other. If the assertion fails, the pointer is converted to an integer type,

and the hexadecimal value of the pointer is printed. For example,

```C++

test(voidPointer) {

  const int aa[] = {1, 2};

  const long bb[] = {1, 2};

  assertEqual(aa, bb);

}

```

This test will fail with the following error message:

```

AUnitTest.ino:338: Assertion failed: (aa=0x3FFFFF38) == (bb=0x3FFFFF30).

Test voidPointer failed.

```

Comparison against the `nullptr` will work:

```C++

test(nullPointer) {

  const int aa[] = {1, 2};

  assertEqual(aa, nullptr);

}

```

This will print the following:

```

AUnitTest.ino:348: Assertion failed: (aa=0x3FFFFF58) == (nullptr=0x0).

Test nullPointer failed.

```

Comparing a string type (i.e. `const char*`, or `const __FlashStringHelper*`)

to a `nullptr` will cause an error due to ambiguous matches on overloaded

functions. The solution is to explicitly cast the `nullptr` to the corresponding

string type:

```C+++

test(stringPointer) {

  const char aa[] = "abc";

  // assertEqual(aa, nullptr); // Causes errors

  assertEqual(aa, (const char*) nullptr); // Works.

}

```



#### Case Insensitive String Comparisons

Two macros provide case-insensitive string comparisons (analogous to

`ASSERT_STRCASEEQ()` and `ASSERT_STRCASENE()` in Google Test):

* `assertStringCaseEqual(a, b)`

* `assertStringCaseNotEqual(a, b)`

The supported types for `(a, b)` are all 9 combinations of Arduino string types:

* `(const char *, const char *)`

* `(const char *, const String&)`

* `(const char *, const __FlashStringHelper*)`

* `(const String&, const char*)`

* `(const String&, const String&)`

* `(const String&, const __FlashStringHelper*)`

* `(const __FlashStringHelper*, const char*)`

* `(const __FlashStringHelper*, const String&)`

* `(const __FlashStringHelper*, const __FlashStringHelper*)`

***ArduinoUnit Compatibility***: _Not available in ArduinoUnit._



### Approximate Comparisons

Floating point values are difficult to compare because of internal rounding

errors. Google Test provides

[two types of macros to handle floating points](https://github.com/google/googletest/blob/master/googletest/docs/advanced.md#floating-point-comparison):

* `ASSERT_FLOAT_EQ(a, b)`, `ASSERT_DOUBLE_EQ(a, b)` - determine if the

  floating point numbers are within 4

  [Units in the Last Place (ULPs)](https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/)

* `ASSERT_NEAR(a, b, error)` - determine if the absolute distance between

  `a` and `b` is within the given `error`

Since floating point operations are relatively rare in Arduino programming,

AUnit offers only the equivalent of `ASSERT_NEAR()` function:

* `assertNear(a, b, error)`

* `assertNotNear(a, b, error)`

Upon failure, the error messages will look something like:

```

AUnitTest.ino:517: Assertion failed: |(1.00) - (1.10)| > (0.20).

AUnitTest.ino:527: Assertion failed: |(4.00) - (1.10)| <= (0.20).

```

Unlike Google Test where `ASSERT_NEAR()` supports only the `double` type, both

`assertNear()` and `assertNotNear()` support integral types as well. The full

list of supported types is:

* `int`

* `unsigned int`

* `long`

* `unsigned long`

* `double`

Other primitive types (e.g. `char` or `float`) will be automatically converted

to one of these supported types by the compiler. (64-bit integer types

`long long` and `unsigned long long` are not supported here.)

Note that the `abs()` of 2 values of a signed integer type can be larger than

the maximum value that can be represented by the given signed type. Since signed

integer overflow is an

[undefined behavior](https://stackoverflow.com/questions/16188263/is-signed-integer-overflow-still-undefined-behavior-in-c)

in C and C++11, I cannot predict what the compiler will do in that case.

Unsigned types should not have this problem because the distance between two

values of an unsigned type should always fit inside the given unsigned type.

Technically, a similar problem exists for the floating point types (which are

naturally signed), but it is unlikely that you are dealing with floating point

values so close to the maximum values.



### Boolean Assertions

The following boolean asserts are also available:

* `assertTrue(condition)`

* `assertFalse(condition)`

***ArduinoUnit Compatibility***: _These are identical to ArduinoUnit._



### Test Fixtures

When the unit tests become more complex, using test fixtures will allow you to

place common data objects and methods into a class that can be shared among

multiple test cases. This concept matches very closely to the test fixtures

in

[Google Test](https://github.com/google/googletest/blob/master/googletest/docs/Primer.md).

To create a test fixture:

1. Derives a new class from either `TestOnce` (if you want to run the test just

   once), or `TestAgain` (if you want to run the test repeatedly).

1. Add any data objects inside the class.

1. Optionally add a `virtual void setup() {...}` method to perform any common

   initialization code. Be sure to call the parent's `setup()` method in the

   *first* line to chain any `setup()` methods defined by the parents. There may

   be multiple parent classes.

1. Optionally add a `virtual void teardown() {...}` method to perform any common

   clean up code. Be sure to call the parent's `teardown()` method in the *last*

   line to chain any `teardown()` methods defined by the parents. There may be

   multiple parent classes.

1. Add any additional shared methods into this new class.

To define your tests, use the `testF()` macro like this:

```C++

class CustomTestOnce: public TestOnce {

  protected:

    // optional

    void setup() override {

      TestOnce::setup();

      ...setup code...

    }

    // optional

    void teardown() override {

      ...teardown code...

      TestOnce::teardown();

    }

    void assertCustomStuff() {

      ...common code...

    }

    int sharedValue;

};

testF(CustomTestOnce, calculate) {

  ...test code here...

}

```

No constructor for `CustomTestOnce` needs to be defined.

Underneath the covers, the `testF()` macro creates a subclass named

`CustomTestOnce_calculate` which inherits from `TestOnce` class. The test code

becomes the body of the `CustomTestOnce_calculate::once()` method. The name of

this test has the class name prepended, so it is `CustomTestOnce_calculate`,

which prevents name collision with other `testF()` tests with the same

name using a different test fixture class.

To define a continuous test, use the `testingF()` macro like this:

```C++

class CustomTestAgain: public TestAgain {

  protected:

    // optional

    void setup() override {

      TestAgain::setup();

      ...setup code...

    }

    // optional

    void teardown() override {

      ...teardown code...

      TestAgain::teardown();

    }

    void assertCustomStuff() {

      ...common code...

    }

    int sharedValue;

};

testingF(CustomTestAgain, calculate) {

  ...test code here...

}

```

Similarly, the `testingF()` macro creates a subclass named

`CustomTestAgain_calculate`, and the test code becomes the body of the

`CustomTestAgain_calculate::again()` method.

See `examples/fixtures/fixtures.ino` to see a working example of the `testF()`

macro.

***ArduinoUnit Compatibility***: _The `testF()` and `testingF()` macros,

and the `teardown()` virtual method are available only in AUnit (and Google

Test), not ArduinoUnit._



### Early Return and Delayed Assertions

AUnit (like ArduinoUnit and Google Test) does not use C++ exceptions. Instead,

the various `assertXxx()` macros perform an early `return` if the condition

evaluates to `false`. That means that the assertions can only bail out of the

current method, not the calling method. If you are using test fixtures, and

create a shared custom assert function, e.g. the `assertCustomStuff()` method

above, any `assertXxx()` statements in shared method will bail out of that

method only. The statement after the `assertCustomStuff()` will continue to

execute.

In other words, in the following example, if the `assertCustomStuff()` fails,

then `assertMoreStuff()` inside `testF()` will execute:

```C++

class CustomTestOnce: public TestOnce {

  protected:

    void assertCustomStuff() {

      assertEqual(sharedValue, 3);

      // This will not execute if the assertEqual() above fails.

      assertLess(...);

    }

    void assertMoreStuff() {

      assertEqual(...);

    }

    int sharedValue;

};

// DON'T DO THIS

testF(CustomTestOnce, dontDoThis) {

  assertCustomStuff();

  // This will execute even if assertCustomStuff() fails.

  assertMoreStuff();

}

// DO THIS INSTEAD

testF(CustomTestOnce, doThis) {

  assertNoFatalFailure(assertCustomStuff());

  assertNoFatalFailure(assertMoreStuff());

}

```

The solution is to use the `assertNoFatalFailure(statement)` macro which checks

whether the inner `statement` returned with a fatal assertion. If so, then it

returns immediately, preventing execution from continuing to the code that

follows. This macro is modeled after the

[ASSERT_NO_FATAL_FAILURE(statement)](https://github.com/google/googletest/blob/master/docs/advanced.md)

macro in Google Test that provides the same functionality.



### Meta Assertions

The following methods from ArduinoUnit have also been implemented:

* `checkTestDone(name)`

* `checkTestNotDone(name)`

* `checkTestPass(name)`

* `checkTestNotPass(name)`

* `checkTestFail(name)`

* `checkTestNotFail(name)`

* `checkTestSkip(name)`

* `checkTestNotSkip(name)`

* `checkTestExpire(name)` [&ast;]

* `checkTestNotExpire(name)` [&ast;]

* `assertTestDone(name)`

* `assertTestNotDone(name)`

* `assertTestPass(name)`

* `assertTestNotPass(name)`

* `assertTestFail(name)`

* `assertTestNotFail(name)`

* `assertTestSkip(name)`

* `assertTestNotSkip(name)`

* `assertTestExpire(name)` [&ast;]

* `assertTestNotExpire(name)` [&ast;]

The `checkTestXxx()` methods check the status of the test named `name`

and returns a `bool`. The execution continues even if `false`.

The `assertTestXxx()` methods stops the unit test if the status check

returns `false`, and prints assertion messages that look like this:

```

AUnitTest.ino:366: Assertion passed: Test slow_pass is done.

AUnitTest.ino:372: Assertion passed: Test slow_pass is not failed.

AUnitTest.ino:448: Assertion passed: Test slow_skip is skipped.

AUnitTest.ino:451: Assertion passed: Test slow_skip is not timed out.

```

(The human readable version of being `expired` will always be `timed out` or

`not timed out` on the `Serial` output.)

The following macros define `extern` references to test case objects which live

in other `.cpp` files. These are required for the above meta assertions if the

test cases are defined in another file:

* `externTest()`

* `externTesting()`

* `externTestF()`

* `externTestingF()`

***ArduinoUnit Compatibility***: _The methods marked by [&ast;] are only

available in AUnit. Also, the assertion messages are different. ArduinoUnit

reuses the format used by the `assertXxx()` macros, so prints something like

the following:_

```

AUnitTest.ino:439: Assertion passed: (test_slow_skip_instance.state=2) >= (Test::DONE_SKIP=2).

```

_AUnit has a separate message handler to print a customized message for the

assertTestXxx() meta assertion macros._



### Unconditional Termination

The following macros can be used inside the body of `test()` or `testing()`

macro to terminate a test unconditionally. Each macro prints a short message,

and returns immediately from the test, much like an `assertXxx()` macro that

fails.

* `passTestNow()`  [&ast;]

* `failTestNow()` [&ast;]

* `skipTestNow()` [&ast;]

* `expireTestNow()` [&ast;]

The messages look like:

```

AUnitTest.ino:360: Status passed.

AUnitTest.ino:378: Status failed.

AUnitTest.ino:380: Status skipped.

AUnitTest.ino:391: Status timed out.

```

The following methods on the `Test` class also set the `status` of the test, but

these methods do not print any messages (which makes debugging difficult) and

they do *not* terminate the test immediately.

* `pass()` - test passed

* `fail()` - test failed

* `skip()` - test skipped

* `expire()`  - test timed out [&ast;]

In most cases, the `failTestNow()`, `skipTestNow()` and `expireTestNow()` macros

are more useful than the equivalent methods in the `Test` class. However, in a

`testing()` loop test, the `pass()` method is probably better than the

`passTestNow()` macro because we usually don't want to see an error message

from a passing test.

***ArduinoUnit Compatibility***:

_The method(s) marked by [&ast;] are only available in AUnit._



### Overridable Methods

The following methods are defined at the `Test` base class level:

* `setup()`

* `teardown()`

The `TestOnce` class defines:

* `once()`

The `TestAgain` class defines:

* `again()`

***ArduinoUnit Compatibility***: _These are functionally the same as ArduinoUnit

except with different class names. Instead of `Test` use `TestAgain`. Instead

of `Test::loop` use `TestAgain::again()`. ArduinoUnit does not support a

`teardown()` method._



### Running the Tests

We run the test cases in the global `loop()` method by calling

`TestRunner::run()`. The tests are sorted according to the name of the test

given in the argument in the `test()` or `testing()` macro.

Each call to the `run()` method causes one test case to run and be resolved. The

next call to `run()` executes the next test case. This design allows the

`loop()` method to perform a small amount of work and return periodically to

allow the system to perform some actions. On some systems, such as the ESP8266,

an error is generated if `loop()` takes too much CPU time.

```C++

...

void loop() {

  TestRunner::run();

}

```

***ArduinoUnit Compatibility***: _This is equivalent to called `Test::run()` in

ArduinoUnit. AUnit sorts the tests in the same way as ArduinoUnit. In

ArduinoUnit, each call to `Test::run()` will process the entire list of

currently active test cases. In AUnit, each call to `TestRunner::run()` performs

only a single test case, then returns._



### Filtering Test Cases

Six filtering methods are available on the `TestRunner` class:

* `TestRunner::include(pattern)` - prefix match

* `TestRunner::include(testClass, pattern)` - prefix match

* `TestRunner::exclude(pattern)` - prefix match

* `TestRunner::exclude(testClass, pattern)` - prefix match

* `TestRunner::includesub(substring)` - substring match (v1.6)

* `TestRunner::excludesub(substring)` - substring match (v1.6)

These methods are called from the global `setup()` method, for example:

```C++

void setup() {

  TestRunner::include("looping*");

  TestRunner::exclude("CustomTestAgain", "*");

  TestRunner::include("CustomTestAgain", "test*");

  TestRunner::include("CustomTestAgain", "test*");

  TestRunner::includesub("net");

  TestRunner::excludesub("net");

  ...

}

```

Excluded tests bypass their `Test::setup()` and `Test::teardown()` methods and

terminate immediately. For the purposes of reporting, excluded tests are counted

as "skipped".

The 2-argument versions of `include()` and `exclude()` correspond to the

2 arguments of `testF()` and `testingF()`.

The filtering methods are also available as command line flags and arguments

(`--include`, `--exclude`, `--includesub` `--excludesub`) if the test

program is compiled using EpoxyDuino under a Unix-like environment. See

the [EpoxyDuino](#EpoxyDuino) section below.

**Implicit Exclude All**: If the *first* filtering request is an "include" (i.e.

`include(pattern)`, `include(testClass, pattern)`, `includesub(substring)`),

all tests are excluded by default initially, instead of being included by

default. Otherwise, the first "include" statement would have no effect.

***ArduinoUnit Compatibility***:

_The equivalent versions in ArduinoUnit are `Test::exclude()` and

`Test::include()` The matching algorithm in AUnit is not as powerful as the one

in ArduinoUnit. AUnit supports only a single wildcard character `*` and that

character can appear only at the end if it is present. For example, the

following are accepted:_

* `TestRunner::exclude("*");`

* `TestRunner::include("f*");`

* `TestRunner::exclude("flash_*");`

* `TestRunner::include("looping*");`

* `TestRunner::include("CustomTestOnce", "flashTest*");`

_AUnit provides 2-argument versions of `include()` and `exclude()`_



### Output Printer

The default output printer is the `Serial` instance. This can be

changed using the `TestRunner::setPrinter()` method:

```C++

#include 

using aunit::TestRunner;

...

void setup() {

  Serial1.begin(...);

  TestRunner::setPrinter(&Serial1);

  ...

}

void loop() {

  TestRunner::run();

}

```

***ArduinoUnit Compatibility***:

_This is the equivalent of the `Test::out` static member variable in

ArduinoUnit._



### Controlling the Verbosity

The default verbosity of the test results can be controlled using the

`TestRunner::setVerbosity()` method:

```C++

#include 

using aunit::TestRunner;

using aunit::Verbosity;

...

void setup() {

  ...

  TestRunner::setVerbosity(Verbosity::kAll);

  ...

}

```

Every test is assigned this default verbosity just before its `Test::setup()`

is called. A unit test can choose to modify the verbosity calling one of the

following methods:

* `void enableVerbosity(uint8_t verbosity);`

    * enables the given verbosity, retaining all the others

* `void disableVerbosity(uint8_t verbosity);`

    * disables the given verbosity, retaining all the others

at the beginning of the test definition, like this:

```C++

test(enable_assertion_passed_messages) {

  enableVerbosity(Verbosity::kAssertionPassed);

  ...

}

```

The values of `verbosity` are defined by the static constants of the `Verbosity`

utility class:

* `Verbosity::kAssertionPassed`

* `Verbosity::kAssertionFailed`

* `Verbosity::kTestPassed`

* `Verbosity::kTestFailed`

* `Verbosity::kTestSkipped`

* `Verbosity::kTestExpired`

* `Verbosity::kTestRunSummary`

* `Verbosity::kAssertionAll` - enables all assert messages

* `Verbosity::kTestAll`

    * same as `(kTestPassed | kTestFailed | kTestSkipped | kTestExpired)`

* `Verbosity::kDefault`

    * same as `(kAssertionFailed | kTestAll | kTestRunSummary )`

* `Verbosity::kAll` - enables all messages

* `Verbosity::kNone` - disables all messages

***ArduinoUnit Compatibility***:

_The following ArduinoUnit variables do not exist:_

* `Test::min_verbosity`

* `Test::max_verbosity`

_The bit field constants have slightly different names:_

* `TEST_VERBOSITY_TESTS_SUMMARY` -> `Verbosity::kTestRunSummary`

* `TEST_VERBOSITY_TESTS_FAILED` -> `Verbosity::kTestFailed`

* `TEST_VERBOSITY_TESTS_PASSED` -> `Verbosity::kTestPassed`

* `TEST_VERBOSITY_TESTS_SKIPPED` -> `Verbosity::kTestSkipped`

* `TEST_VERBOSITY_TESTS_ALL` -> `Verbosity::kTestAll`

* `TEST_VERBOSITY_ASSERTIONS_FAILED` -> `Verbosity::kAssertionFailed`

* `TEST_VERBOSITY_ASSERTIONS_PASSED` -> `Verbosity::kAssertionPassed`

* `TEST_VERBOSITY_ASSERTIONS_ALL` -> `Verbosity::kAssertionAll`

* `TEST_VERBOSITY_ALL` -> `Verbosity::kAll`

* `TEST_VERBOSITY_NONE` -> `Verbosity::kNone`

* {no equivalent} <- `Verbosity::kDefault`

* {no equivalent} <- `Verbosity::kTestExpired`



### Line Number Mismatch

AUnit suffers from the same compiler/preprocessor bug as ArduinoUnit that causes

the built-in `__LINE__` macro to be off by one. The solution is to add:

```C++

#line 2 "filename.ino"

```

as the first line of a unit test sketch, where the "filename.ino" must appear in

double-quotes.

***ArduinoUnit Compatibility***: _This problem is identical to ArduinoUnit._



### Test Framework Messages



#### Assertion Message

The various `assertXxx()` macros in AUnit print a message upon pass or fail. For

example, if the assertion was:

```C++

int expected = 3;

int counter = 4;

assertEquals(expected, counter);

```

The error message (if enabled, which is the default) is:

```

AUnitTest.ino:134: Assertion failed: (3) == (4).

```

Asserts with `bool` values produce customized messages, printing "true" or

"false" instead of using the Print class default conversion to `int`:

```C++

assertEquals(true, false);

AUnitTest.ino:134: Assertion failed: (true) == (false).

```

Similarly, the `assertTrue()` and `assertFalse()` macros provide more customized

messages:

```C++

bool ok = false;

assertTrue(ok);

AUnitTest.ino:134: Assertion failed: (false) is true.

```

and

```C++

bool ok = true;

assertFalse(ok);

AUnitTest.ino:134: Assertion failed: (true) is false.

```

***ArduinoUnit Compatibility***:

_ArduinoUnit captures the arguments of the `assertEqual()` macro

and prints:_

```

AUnitTest.ino:134: Assertion failed: (expected=3) == (counter=4).

```

_Each capture of the parameter string consumes flash memory space. If the unit

test has numerous `assertXxx()` statements, the flash memory cost is expensive.

AUnit omits the parameters to reduce flash memory space by about 33%._

_The messages for asserts with bool values are customized for better clarity

(partially to compensate for the lack of capture of the string of the actual

arguments, and are different from ArduinoUnit._



#### Verbose Mode Message

If you use the verbose header:

```C++

#include 

```

the assertion message will contain the string fragments of the arguments

passed into the `assertXxx()` macros, like this:

```

AUnitTest.ino:134: Assertion failed: (expected=3) == (counter=4).

AUnitTest.ino:134: Assertion failed: (ok=false) is true.

```

instead of:

```

AUnitTest.ino:134: Assertion failed: (3) == (4).

AUnitTest.ino:134: Assertion failed: (false) is true.

```

***ArduinoUnit Compatibility***:

_As of v1.7, the assertion message format is compatible with the vim editor

and other Linux/MacOS/Unix tools, and no longer compatible with ArduinoUnit



#### Test Case Summary

As each test case finishes, the `TestRunner` prints out the summary of the test

case like this:

```

Test bad failed.

Test looping_pass passed.

Test looping_skip skipped.

Test looping_until timed out.

```

***ArduinoUnit Compatibility***: _These are identifcal to ArduinoUnit,

except that the "timed out" status is new to AUnit. See Test Timeout

section below._



#### Test Runner Summary

At the end of the test run, the `TestRunner` prints out the summary

of all test cases, like this:

```

TestRunner duration: 0.05 seconds.

TestRunner summary: 12 passed, 0 failed, 2 skipped, 1 timed out, out of 15 test(s).

```

***ArduinoUnit Compatibility***: _The message format is slightly different than

ArduinoUnit. I changed "Test summary" to "TestRunner summary" because the former

looks identical to the message that could have been printed by a `test(summary)`

test case. AUnit also adds information about tests which timed out. See below._



### Test Timeout

***ArduinoUnit Compatibility***: _Only available in AUnit._

From my experience, it seems incredibly easy to write a `testing()` test case

which accidentally runs forever because the code forgets to call an explicit

`pass()`, `fail()` or `skip()`.

The `TestRunner` in AUnit applies a timeout value to all the test cases that it

runs. The **default timeout is 10 seconds**. A timeout value of `0` means an

**infinite** timeout, which means that the `testing()` test case may run

forever. The value of the timeout is stored as a `uint16_t` type, so the maximum

timeout is 65535 seconds or a bit over 18 hours.

Currently, the timeout value is global to all test cases. Time out values for

individual tests cannot be set independently. If a test does not finish before

that time, then the test is marked as `timed out` (internally implemented by the

`Test::expire()` method) and a message is printed like this:

```

Test looping_until timed out.

```

The timeout value can be changed by calling the static

`TestRunner::setTimeout()` method. Here is an example that sets the timeout to

30 seconds instead:

```C++

void setup() {

  ...

  TestRunner::setTimeout(30);

  ...

}

```

***ArduinoUnit Compatibility***: _Only available in AUnit._



## GoogleTest Adapter

It may be possible to run simple unit tests written using

[Google Test](https://github.com/google/googletest/) API on an Arduino platform

by using the

[aunit/contrib/gtest.h](src/aunit/contrib/gtest.h) adapter. This

adapter layer provides a number of Google Test macros which map to

their equivalent macros in AUnit:

* `ASSERT_EQ(e, a)` - `assertEqual()`

* `ASSERT_NE(e, a)` - `assertNotEqual()`

* `ASSERT_LT(e, a)` - `assertLess()`

* `ASSERT_GT(e, a)` - `assertMore()`

* `ASSERT_LE(e, a)` - `assertLessOrEqual()`

* `ASSERT_GE(e, a)` - `assertMoreOrEqual()`

* `ASSERT_STREQ(e, a)` - `assertEqual()`

* `ASSERT_STRNE(e, a)` - `assertNotEqual()`

* `ASSERT_STRCASEEQ(e, a)` - `assertStringCaseEqual()`

* `ASSERT_STRCASENE(e, a)` - `assertStringCaseNotEqual()`

* `ASSERT_TRUE(x)` - `assertTrue()`

* `ASSERT_FALSE(x)` - `assertFalse()`

* `ASSERT_NEAR(e, a, error)` - `assertNear()`

To use the `gtest.h` adapter, include the following headers:

```C++

#include 

#include 

```

or

```C++

#include 

#include 

```



## Command Line Tools

Each unit test is an independent `*.ino` program. You can run it using your

Arduino IDE. But there are 2 command line tools that can be used to run them.



### AUniter

The `auniter.sh` script used to be part of this project, but now lives in

its own AUniter (https://github.com/bxparks/AUniter) project. The `auniter.sh`

script is a wrapper around:

* [Arduino IDE in command line mode](https://github.com/arduino/Arduino/blob/master/build/shared/manpage.adoc), and

* [Arduino CLI tool](https://github.com/arduino/arduino-cli)

Using `auniter.sh`, you can compile, upload and validate multiple AUnit tests on

multiple Arduino boards using a single command.

The script can monitor the serial port and determine if the unit test passed or

failed, and it will print out a summary of all unit tests at the end. Full

details are given in the AUniter project, but here are some quick examples

copied from the `AUniter/README.md` file:

* `$ auniter envs`

    * list the environments configured in the `auniter.ini` config file

* `$ auniter ports`

    * list the available serial ports and devices

* `$ auniter verify nano Blink.ino`

    * verify (compile) `Blink.ino` using the `env:nano` environment

* `$ auniter upload nano:/dev/ttyUSB0 Blink.ino`

    * upload `Blink.ino` to the `env:nano` target environment connected to

    `/dev/ttyUSB0`

* `$ auniter test nano:USB0 BlinkTest.ino`

    * compile and upload `BlinkTest.ino` using the `env:nano` environment,

      upload it to the board at `/dev/ttyUSB0`, then validate the output of the

      [AUnit](https://github.com/bxparks/AUnit) unit test

* `$ auniter test nano:USB0 BlinkTest/ ClockTest/`

    * upload and verify the 2 unit tests (`BlinkTest/BlinkTest.ino`,

      `ClockTest/ClockTest.ino`) on the target environment (`env:nano`)

* `$ auniter upmon nano:USB0 Blink.ino`

    * upload the `Blink.ino` sketch and monitor the serial port using a

      user-configurable terminal program (e.g. `picocom`) on `/dev/ttyUSB0`



### EpoxyDuino

Instead of running the unit tests on the actual microcontrollers themselves, you

can compile and execute AUnit unit tests natively on Linux or MacOS machines

using the EpoxyDuino (https://github.com/bxparks/EpoxyDuino) project.

EpoxyDuino provides a minimal Arduino programming environment that is usually

sufficient to compile and run AUnit test units on the Unix host machine. It

relies on the native C++ compiler, GNU Make, and `Makefile` files for each

`*.ino` unit test like this:

```

APP_NAME := SampleTest

ARDUINO_LIBS := AUnit

include ../../../EpoxyDuino/EpoxyDuino.mk

```

The unit test is compiled into a binary (`SampleTest.out`) using the `make

command, and the binary can be executed like this:

```

$ make

$ ./SampleTest.out

```

Here are a few tips when writing unit tests to run under EpoxyDuino:

**Delay(1000)**

For real Arduino boards, you get more reliable unit tests if you add a

`delay(1000)` at the start of the program. For EpoxyDuino, this is not

necessary, so I recommend calling this only on real Arduino boards, like this:

```C++

void setup() {

#if ! defined(EPOXY_DUINO)

  delay(1000); // Wait for stability on some boards, otherwise garage on Serial

#endif

  Serial.begin(115200);

  while (! Serial); // Wait until Serial is ready - Leonardo/Micro

#if defined(EPOXY_DUINO)

  Serial.setLineModeUnix(); // use Unix line terminator instead of DOS

#endif

  ...

}

void loop() {

  aunit::TestRunner::run();

}

```

**Exit() Status Code**

On real Arduino boards, the unit test (or any program for that matter) never

terminates. The `loop()` function executes forever. On Linux or MacOS using

EpoxyDuino, the test program will terminate at the end through the

`exit()` function. If the tests are successful (i.e. passing or skipped), it

will call `exit(0)`. If there are any failing tests (i.e. failed or timed out),

it will call `exit(1)`.



### Command Line Flags and Arguments

(Added in v1.6)

The standard Arduino environment does not provide command line arguments, since

a microcontroller does not normally provide a command line environment. However,

if the AUnit test program is compiled under EpoxyDuino, the standard Unix

command line parameters (`argc` and `argv`)  become available through the

`extern int epoxy_argc` and `extern const char* const* epoxy_argv` global

variables. These allow the `TestRunner` class to provide command line flags and

arguments as follows:

```bash

$ ./test.out --help

Usage: ./test.out [--help] [--include pattern,...] [--exclude pattern,...]

   [--includesub substring,...] [--excludesub substring,...]

   [--] [substring ...]

```

Example, the following runs all tests with substring "net" or "led" in its

name, and skips all others:

```bash

$ ./test.out net led

```

Flags:

* `--include pattern,...`

    * Comma-separated list of patterns to pass to the

      `TestRunner::include(pattern)` method

* `--exclude pattern,...`

    * Comma-separated list of patterns to pass to the

      `TestRunner::exclude(pattern)` method

* `--includesub substring,...`

    * Comma-separated list of substrings to pass to the

      `TestRunner::includesub(substring)` method

* `--excludesub substring,...`

    * Comma-separated list of substrings to pass to the

      `TestRunner::excludesub(substring)` method

Arguments:

* Any **Space**-separated list of words after the optional flags are passed to

  the `TestRunner::includesub(substring)` method.

The command line flags and arguments are processed *after* any hardcoded calls

to `TestRunner::include()` and `TestRunner::exclude()` methods in the global

`setup()` method.

The flags and command line arguments are processed *in order* of appearance

on the command line.

Similar to the hardcoded calls to `TestRunner::include()` and

`TestRunner::exclude()`, if the first command line flag is an `--include` or

`--includesub`, then all tests are *excluded* by default initially. Otherwise,

the first include flag would have no effect.



## Continuous Integration

There are several ways to incorporate AUnit into a continuous integration

system. At the infrastructure level, you can use either a system like

[Jenkins](https://jenkins.io) running on a local machine, or use a cloud-based

continuous integration system like [GitHub

Actions](https://github.com/features/actions). For each of those

infrastructures, you can choose to use the Arduino IDE or CLI build tools, or

you can use EpoxyDuino to compile and run against a Linux or MacOS

environment.

The option matrix looks like this:

```

+----------------+-------------------------+------------------------+

|\ CI environment|                         |                        |

| \____________  | Local (e.g. Jenkins)    | Cloud (e.g. GitHub)    |

|              \ |                         |                        |

| Build tool    \|                         |                        |

+----------------+-------------------------+------------------------+

|                | * can execute tests     | * verify compile-only  |

|                |   on microcontroller    | * cannot actually run  |

| Arduino        | * complex setup and     |   tests on             |

| IDE/CLI        | * maintenance           |   microcontroller      |

|                | * can be slow           | * complex installation |

|                | * (not recommended)     |   and setup            |

|                |                         | * (unverified)         |

+----------------+-------------------------+------------------------+

|                | * verify execution on   | * verify execution on  |

|                |   Unix environment      |   Unix environment     |

| EpoxyDuino     | * faster than IDE/CLI   | * simple setup and     |

|                | * complex setup and     |   maintenance          |

|                |   and maintenance       | * very fast            |

|                | * (unnecessary)         | * (recommended)        |

+----------------+-------------------------+------------------------+

```



### Arduino IDE/CLI + Cloud

The big advantage of using the Arduino IDE/CLI to run the AUnit unit tests is

that you can compile them using the precise compiler and tool chain that will be

used against the specific microcontroller that you are interested in verifying.

There can be subtle compiler differences (e.g. size of `int`) or differences in

how the Arduino programming environment was implemented (e.g. availability of

the `FPSTR()` macro). If you use a cloud-based CI infrastructure, then you must

install the complete Arduino IDE/CLI environment into your cloud test runner.

Although I think it's theoretically possible, I have never actually verified

that this can be done.



### Arduino IDE/CLI + Jenkins

This setup is described in [Continuous Integration with

Jenkins](https://github.com/bxparks/AUniter/tree/develop/jenkins), and it worked

reasonably well for small number of unit tests. The problem is that the Arduino

IDE is far too slow when the number of unit tests become non-trivial. And it

also takes too much effort to maintain the local Jenkins infrastructure. The

Jenkins environment seems to be brittle due to its complexity of all of its

moving parts. This is the only environment where you can connect a real Arduino

microcontroller to the local machine and have the unit tests run on the actual

microcontroller. Perhaps for certain situations, running the unit tests on

actual hardware is a requirement. But for most people, I no longer recommend

this environment.



### EpoxyDuino + Jenkins

Once the Jenkins environment is up and running, I have verified that it is easy

to run the unit tests using EpoxyDuino, since it needs just a C++ compiler

and GNU Make. Things will compile and run a lot faster than using the Arduino

IDE/CLI. However, this combination suffers from the same problem of maintaining

the Jenkins environment. If the unit tests are running in an Unix environment

anyway, it seems far easier to just run them in the cloud. So I don't recommend

using this setup. Just use a cloud CI provider as described below.



### EpoxyDuino + Cloud (Recommended)

A cloud-based continuous integration service like [GitHub

Actions](https://github.com/features/actions) is easy to setup for

EpoxyDuino. Often the C++ compiler and GNU `make` tools are already installed

in the Docker container used by the CI system. The only additional setup is

to install EpoxyDuino, AUnit and other dependent Arduino libraries.

Here are some example YAML files for GitHub Actions:

* https://github.com/bxparks/AceTime/tree/develop/.github/workflows

* https://github.com/bxparks/AceButton/tree/develop/.github/workflows

* https://github.com/bxparks/AceRoutine/tree/develop/.github/workflows

* https://github.com/bxparks/AceCRC/tree/develop/.github/workflows

In the various `aunit_tests.yml` files, the `Setup` step installs the various

dependent libraries using the `git clone` command, for example:

```

git clone https://github.com/bxparks/EpoxyDuino

git clone https://github.com/bxparks/AUnit

```

These commands install the default branch for those repositories, which for most

of my libraries will be the `develop` branch. This makes sense for me because I

want the unit tests to run against the latest commits. However, for many others,

it is probably better to use the `master` branch because it contains the stable

releases:

```

git clone --branch master https://github.com/bxparks/EpoxyDuino

git clone --branch master https://github.com/bxparks/AUnit

```

Using EpoxyDuino with a cloud CI provider (like GitHub Actions) is my

recommended configuration for running AUnit tests because it is easy to setup

and maintain and the tests run fast.



## Tips

Collection of useful tidbits.



### Debugging Assertions in Fixtures

When using test fixtures with the `testF()` and `testingF()` macros, it's often

useful to create helper assertions, such as the `assertCustomStuff()` below.

Debugging such assertion statements can be tricky. I've found that turning on

messages for successful assertions (with a

`enableVerbosity(Verbosity::kAssertionPassed)`) statement can be very helpful:

```C++

class CustomTestOnce: public TestOnce {

  protected:

    // optional

    void setup() override {

      TestOnce::setup();

      ...setup code...

    }

    // optional

    void teardown() override {

      ...teardown code...

      TestOnce::teardown();

    }

    void assertCustomStuff() {

      assertEqual(...);

      ...

      for (...) {

        ...

        assertEqual(...);

        ...

      }

    }

    int sharedValue;

};

testF(CustomTestOnce, calculate) {

  enableVerbosity(Verbosity::kAssertionPassed);

  ...test code here...

  assertCustomStuff();

}

```



### Class Hierarchy

To support test fixtures in a more natural way, the class hierarchy

in AUnit is slightly different than ArduinoUnit. In ArduinoUnit we have a

two level hierarchy:

```

  Test ::loop()

    ^

    |

 TestOnce ::once()

 ```

In AUnit, the functionality that supports the `testing()` macro has been

migrated to a separate class called `TestAgain`, like this:

```

       Test ::loop()

         ^

         |

      Assertion

         ^

         |

     MetaAssertion

        ^  ^

       /    \

      /      \

TestAgain  TestOnce

::again()  ::once()

```

Normally, deep inheritance hierarchies like this should be avoided. However,

placing the `Assertion` and `MetaAssertion` classes inside the `Test` hierarchy

allowed those assertion statements to have access to the internal states of the

`Test` instance. This made certain features (like the early return upon delayed

failure) slightly easier to implement. For the most part, the end-users can

ignore the existence of the `Assertion` and `MetaAssertion` classes and think of

this as a simple 2-level inheritance tree.



### Testing Private Helper Methods

There is a school of thought which says that unit tests should test only the

publically exposed methods of a class or library. I agree mostly with that

sentiment, but not rigidly. I think it is sometimes useful to write unit tests

for `protected` or `private` methods. For example, when creating a chain of

small helper methods, which build up to larger publically exposed methods, it is

extremely useful to write unit tests for the helper methods in isolation.

Normally those helper methods would be `private` because they are used

only within that class, and we don't want to expose them to the public API. One

option is to make them `public` but add a comment in the function to say that it

is exposed only for testing purposes. This does not seem satisfactory because

users will tend to ignore such comments if the helper functions are useful.

I think a better way is to keep the helper functions `private` but make

the unit tests a `friend class` of the target class. The syntax for doing this

can be tricky, it took me a number of attempts to get this right, especially if

when using namespaces for the target class:

```C++

//------------------- Target.h -------------

// Auto-generated test class names.

class Test_helper;

class TargetSuite_helper;

class TargetTest_helper;

namespace mylib {

class Target {

  public:

    void publicMethod() {

      ...

      int a = helper();

      ...

    }

  private:

    // Must have the global scope operator '::'

    friend class ::Test_helper;

    friend class ::TargetSuite_helper;

    friend class ::TargetTest_helper;

    static int helper() {...}

};

}

//------------------- TargetTest.ino -------------

#include 

#include "Target.h"

using namespace aunit;

using namespace mylib;

test(helper) {

  assertEqual(1, Target::helper(...));

}

test(TargetSuite, helper) {

  assertEqual(1, Target::helper(...));

}

class TargetTest: public TestOnce {

  ...

};

testF(TargetTest, helper) {

  assertEqual(1, Target::helper(...));

}

```

The tricky part is that `Target.h` must have forward declarations of the various

auto-generated AUnit test classes. And within the `Target` class itself, the

`friend` declarations need to have a global scope `::` specifier before the name

of the test class.



## Benchmarks



### MemoryBenchmark

The [MemoryBenchmark](examples/MemoryBenchmark/) directory collects the flash

and static RAM usage of the AUnit library on various microcontroller boards, and

renders the results as tables embedded in the README.md file. Here are 2

highlights:

**Arduino Nano (8-bit)**

```

+---------------------------------------------------------------------+

| Functionality                          |  flash/  ram |       delta |

|----------------------------------------+--------------+-------------|

| Baseline                               |   1586/  185 |     0/    0 |

|----------------------------------------+--------------+-------------|

| AUnit Single Test                      |   4456/  366 |  2870/  181 |

| AUnit Single Test Verbose              |   4500/  366 |  2914/  181 |

+---------------------------------------------------------------------+

```

**ESP8266 (32-bit)**

```

+---------------------------------------------------------------------+

| Functionality                          |  flash/  ram |       delta |

|----------------------------------------+--------------+-------------|

| Baseline                               | 264949/27984 |     0/    0 |

|----------------------------------------+--------------+-------------|

| AUnit Single Test                      | 268021/28148 |  3072/  164 |

| AUnit Single Test Verbose              | 268081/28148 |  3132/  164 |

+---------------------------------------------------------------------+

```



### Compared to ArduinoUnit 2.2

AUnit consumes as much as 65% less flash memory than ArduinoUnit 2.2 on an AVR

platform (e.g. Arduino UNO, Nano), and 30% less flash on the Teensy-ARM platform

(e.g. Teensy LC ). (ArduinoUnit 2.3 reduces the flash memory by 30% or so, which

means that AUnit can still consume significantly less flash memory.)

Here are the resource consumption (flash and static) numbers

from

[AceButtonTest](https://github.com/bxparks/AceButton/tree/develop/tests/AceButtonTest)

containing 26 test cases using 331 `assertXxx()`

statements, compiled using AUnit and ArduinoUnit 2.2 on 5 different

microcontrollers:

```

---------------------------+---------+-------------+-------------+

Platform (resource)        |     Max | ArduinoUnit |       AUnit |

---------------------------+---------+-------------+-------------|

Arduino Nano (flash)       |   30720 |       54038 |       18928 |

Arduino Nano (static)      |    2048 |        1061 |         917 |

---------------------------+---------+-------------+-------------|

Teensy LC (flash)          |   63488 |       36196 |       26496 |

Teensy LC (static)         |    8192 |        2980 |        2780 |

---------------------------+---------+-------------+-------------|

Teensy 3.2 (flash)         |  262144 |       51236 |       37920 |

Teensy 3.2 (static)        |   65536 |        5328 |        5236 |

---------------------------+---------+-------------+-------------|

ESP8266 - ESP-12E (flash)  | 1044464 |    does not |      268236 |

ESP8266 - ESP-12E (static) |   81920 |     compile |       33128 |

---------------------------+---------+-------------+-------------|

ESP8266 - ESP-01 (flash)   |  499696 |    does not |      268236 |

ESP8266 - ESP-01 (static)  |   47356 |     compile |       33128 |

---------------------------+---------+-------------+-------------+

```

Not all unit test sketches will experience a savings of 65% of flash memory with

AUnit, but a savings of 30-50% seems to be common.



## System Requirements



### Hardware

**Tier 1: Fully supported**

These boards are tested on each release:

* Arduino Nano clone (16 MHz ATmega328P)

* SparkFun Pro Micro clone (16 MHz ATmega32U4)

* Seeeduino XIAO M0 (SAMD21, 48 MHz ARM Cortex-M0+)

* STM32 Blue Pill (STM32F103C8, 72 MHz ARM Cortex-M3)

* Adafruit ItsyBitsy M4 (SAMD51, 120 MHz ARM Cortext-M4)

* NodeMCU 1.0 (ESP-12E module, 80 MHz ESP8266)

* WeMos D1 Mini (ESP-12E module, 80 MHz ESP8266)

* ESP32 dev board (ESP-WROOM-32 module, 240 MHz dual core Tensilica LX6)

**Tier 2: Should work**

These boards should work but I don't test them as often:

* Arduino Pro Mini (16 MHz ATmega328P)

* Mini Mega 2560 (Arduino Mega 2560 compatible, 16 MHz ATmega2560)

* Teensy LC (48 MHz ARM Cortex-M0+)

* Teensy 3.2 (96 MHz ARM Cortex-M4)

**Tier 3: May work, but not supported**

* Other 3rd party SAMD21 and SAMD51 boards *may* work if their board software

  uses the traditional Arduino API, instead of the

  [ArduinoCore-API](https://github.com/arduino/ArduinoCore-api)

**Tier Blacklisted**

The following boards are *not* supported and are explicitly blacklisted to allow

the compiler to print useful error messages instead of hundreds of lines of

compiler errors:

* Any platform using the

    [ArduinoCore-API](https://github.com/arduino/ArduinoCore-api), such as:

    * Arduino Nano Every

    * Arduino Nano 33 IoT

    * Arduino MKRZero

    * Arduino UNO R4

    * Raspberry Pi Pico (RP2040)



### Toolchains

This library was validated using:

* [Arduino IDE 1.8.19](https://www.arduino.cc/en/Main/Software)

* [Arduino CLI 0.33.0](https://arduino.github.io/arduino-cli)

* [Arduino AVR Boards 1.8.6](https://github.com/arduino/ArduinoCore-avr)

* [Arduino SAMD Boards 1.8.9](https://github.com/arduino/ArduinoCore-samd)

  (versions >= 1.8.10 not supported)

* [SparkFun AVR Boards 1.1.13](https://github.com/sparkfun/Arduino_Boards)

* [SparkFun SAMD Boards 1.8.6](https://github.com/sparkfun/Arduino_Boards)

* [STM32duino 2.5.0](https://github.com/stm32duino/Arduino_Core_STM32)

* [ESP8266 Arduino 3.0.2](https://github.com/esp8266/Arduino)

* [ESP32 Arduino 2.0.9](https://github.com/espressif/arduino-esp32)

* [Teensyduino 1.57](https://www.pjrc.com/teensy/td_download.html)

This library is *not* compatible with any platform using the

[ArduinoCore-API](https://github.com/arduino/ArduinoCore-api), such as:

* [ArduinoCore SAMD >=1.8.10](https://github.com/arduino/ArduinoCore-samd)

* [ArduinoCore megaAVR](https://github.com/arduino/ArduinoCore-megaavr/)

* [ArduinoCore renesas](https://github.com/arduino/ArduinoCore-renesas)

* [Arduino-Pico](https://github.com/earlephilhower/arduino-pico)

* [MegaCoreX](https://github.com/MCUdude/MegaCoreX)

(See [Issue #56](https://github.com/bxparks/AUnit/issues/56)

and [Issue #66](https://github.com/bxparks/AUnit/issues/66)).

It should work with [PlatformIO](https://platformio.org/) but I have

not tested it extensively.

The library works under Linux or MacOS (using both g++ and clang++ compilers)

using the EpoxyDuino (https://github.com/bxparks/EpoxyDuino) emulation layer.



### Operating System

I use Ubuntu 20.04 for the vast majority of my development. I expect that the

library will work fine under MacOS and Windows, but I have not tested them.



## License

[MIT License](https://opensource.org/licenses/MIT)



## Feedback and Support

If you have any questions, comments, or feature requests for this library,

please use the [GitHub

Discussions](https://github.com/bxparks/AUnit/discussions) for this project. If

you have bug reports, please file a ticket in [GitHub

Issues](https://github.com/bxparks/AUnit/issues). Feature requests should go

into Discussions first because they often have alternative solutions which are

useful to remain visible, instead of disappearing from the default view of the

Issue tracker after the ticket is closed.

Please refrain from emailing me directly unless the content is sensitive. The

problem with email is that I cannot reference the email conversation when other

people ask similar questions later.



## Authors

* Created by Brian T. Park ([email protected]).

* The Google Test adapter (`gtest.h`) was created by Chris Johnson

  ([email protected]).

* @brewmanz increased the maximum allowed value of `TestRunner::setTimeout()`

  from 255 seconds to 65535 seconds (18.2 hours). (See [Issue

  #57](https://github.com/bxparks/AUnit/issues/57)).

* The design and syntax of many macros (e.g. `test()`, `assertXxx()`) were

  borrowed from the [ArduinoUnit](https://github.com/mmurdoch/arduinounit)

  project to allow AUnit to be almost a drop-in replacement. Many thanks to

  the ArduinoUnit team for creating such an easy-to-use API.