https://github.com/mulle-sde/mulle-test
☑︎ Runs tests and compares results against expectations
https://github.com/mulle-sde/mulle-test
mulle-sde testing
Last synced: 6 months ago
JSON representation
☑︎ Runs tests and compares results against expectations
- Host: GitHub
- URL: https://github.com/mulle-sde/mulle-test
- Owner: mulle-sde
- License: bsd-3-clause
- Created: 2018-01-03T20:29:29.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2026-01-13T00:43:22.000Z (6 months ago)
- Last Synced: 2026-01-13T04:37:34.918Z (6 months ago)
- Topics: mulle-sde, testing
- Language: Shell
- Homepage:
- Size: 737 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# ☑︎ Run tests and compare their results against expectations
... for Android, BSDs, Linux, macOS, SunOS, Windows (MinGW, WSL)
**mulle-test** can test your C or Objective C library. But it can also test
shell scripts or binaries. Basically anything that creates text output.
The loop **mulle-test** performs is *compile*, *execute*, *diff*. A test always
fails, if the return code is not 0. A test also fails, if there is an expected
output file present, and the test output differs from it.
As you would manually:
``` bash
cc -o a.out a.c
./a.out > a.result
diff a.stdout a.result
```
| Release Version | Release Notes
|-------------------------------------------------------|--------------
|  | [RELEASENOTES](RELEASENOTES.md) |
## Writing a test
Here is a simple test, that checks that "Hello World" is properly output.
It consists of a sourcefile `example.c` and the expected test output file
`example.stdout`:
``` bash
mkdir test
cd test
mkdir example
cat < example/example.c
#include
main()
{
printf( "Hello World\n");
return( 0); // important!
}
EOF
echo "Hello World" > example/example.stdout
```
## Testing with test support files
Run the test with `mulle-test`. The following graphic represents the control
flow and the operations performed to check if a test (in this case named "a.c")
passes or fails:
Test support files are optional. If a file is not present, the respective
matching operation is not performed.
| Extension | Description
|----------------------|----------------------
| `.stdout` | Contents must match exactly standard output of the test executable
| `.stdin` | File fed to standard input of the test executable
| `.stderr` | Contents must match exactly standard error of the test executable
| `.errors` | Each line must grep for test diagnostics
| `.ccdiag` | Each line must grep for compiler diagnostics
| `.environment` | Environment variables to set for test executable
| `.cat` | An executable or script to use instead of the default `cat`
| `.diff` | An executable or script to use instead of the default `diff`
If for a test there is no support file `.` found, a file with the
name `default.` will be searched for instead.
### stdin, stdout, stderr
These files are treated as binaries. Changes in whitespace will be detected,
and will produce a warning.
### ccdiag
Each line is used as a pattern in an `grep -E` statement. An example file to
expect a compiler warning could be:
```
may not respond to
```
### errors
Usually a test that returns non-nil is assumed to have failed. If an `errors`
file is present, failure is the expected outcome. The contents of the file
are `grep -E`ed with the test output.
### environment
The contents of this file are loaded before executing the test.
Example:
``` bash
export FOO="1848"
```
### cat
This is an executable or script, that is used to massage output of the
test for diffing. Typical use is to sort the output to avoid differences due
to multi-threading or removing values that look like addresses.
### diff
This is an executable or script, that is used to compare the output of the
test with the expected files `stdout` and `stderr`. Typical use is to remove
some content before diffing such as:
```
#! /bin/sh
# remove all lines starting from "__DATE__" and ending with ");"
sed -e '/__DATE__/,/);/d' | diff "$@"
```
## Technical info
**mulle-test** is based on **mulle-sde**. mulle-test compiles your project and
all dependencies as *shared* libraries. This is different from **mulle-sde**,
which compiles *static* libraries by default.
## Install
See [mulle-sde-developer](//github.com/mulle-sde/mulle-sde-developer) how to
install mulle-sde, which will also install mulle-test with required
dependencies.
The command to install only the latest mulle-test into
`/usr/local` (with **sudo**) is:
``` bash
curl -L 'https://github.com/mulle-sde/mulle-test/archive/latest.tar.gz' \
| tar xfz - && cd 'mulle-test-latest' && sudo ./bin/installer /usr/local
```
## Author
[Nat!](https://mulle-kybernetik.com/weblog) for Mulle kybernetiK