https://github.com/box/makefile.test
A makefile used for running test executables
https://github.com/box/makefile.test
Last synced: 11 months ago
JSON representation
A makefile used for running test executables
- Host: GitHub
- URL: https://github.com/box/makefile.test
- Owner: box
- License: apache-2.0
- Created: 2017-11-30T00:29:40.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2018-01-15T08:06:37.000Z (over 8 years ago)
- Last Synced: 2025-05-06T14:37:22.568Z (about 1 year ago)
- Language: Python
- Homepage: https://blog.box.com/blog/introducing-makefiletest-generic-makefile-run-test-executables/
- Size: 22.5 KB
- Stars: 33
- Watchers: 4
- Forks: 12
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# Makefile.test
[](http://opensource.box.com/badges)
[](https://circleci.com/gh/box/Makefile.test)
A makefile used for running test executables.
Makefile.test can be used to run any type of test executables. It is not
language specific nor it requires any changes to your code. Parallel, serial
execution, various platforms and make versions are supported. The executables
can be organized in any desired way. The user only lists the test files, the
rest is taken care of Makefile.test.
Makefile.test does not contain any rules for compilation and other
pre-processing steps. If your test executables are not scripts, but for example
compiled binaries, you will need to extend Makefile.test with additional rules.
Makefile.test can still be a good starting point for those scenarios.
Makefile.test runs on a single host and therefore its parallelization is
limited with the resources of one machine. If your test suite requires multiple
hosts to run, [ClusterRunner](http://www.clusterrunner.com/) can be a better
tool for your use case.
## Usage:
### Example: A repo that has a `src` and a `test` directory.
A simple repository has a `src` and a `test` directory at its root. The
programmer places application code in `src` and test executables in `test`.
Using the `Makefile.test`, the executables in `test` can be executed with ease.
The directory structure can look like the following:
```
ExampleRepo
├── Makefile.test
├── src
│ └── ExampleApplication.sh
└── test
├── ExampleTest1.sh
├── ExampleTest2.py
└── Makefile
```
> For a recommended way to place the Makefile.test into your own repo see the
> [Installation](#installation) section next.
The `Makefile` file in the `test` directory needs to list the executables in a
`TESTS` variable and include the `Makefile.test`
```
TESTS ?= \
ExampleTest1.sh \
ExampleTest2.py
include ../Makefile.test
```
To run the tests, any of the following can be used from the repo root.
```
cd test && make -j
make -C test -j
make -f test/Makefile -j
```
The output looks as follows:
```
[ExampleTest1.sh] Running ExampleTest1
[ExampleTest2.py] Running ExampleTest2
PASSED: ExampleTest1.sh
PASSED: ExampleTest2.py
---------------------------------
All 2 tests passed
---------------------------------
```
### Running one test at a time.
During development or debugging time, you may want to execute only one test at
a time. In order to achive that without modifying any files, overwrite the
TESTS environment variable from the command line:
```
TESTS=ExampleTest2.py make
```
Only runs the specified test:
```
[ExampleTest2.py] Running ExampleTest2
PASSED: ExampleTest2.py
---------------------------------
All 1 tests passed
---------------------------------
```
## Installation:
### Requirements
- [`bash`](https://www.gnu.org/software/bash/) needs to be installed at `/bin/bash`.
### Using git submodules and symlink to the Makefile.test.
In the directory you want to place Makefile.test execute the following:
```
git submodule add https://github.com/box/Makefile.test.git .Makefile.test
ln -s .Makefile.test/Makefile.test
```
First command creates a hidden dir with the submoduled repo.
Second command symlinks the Makefile.test file.
The directory tree of `ExampleRepo` with the submodule and the symlink looks like
this:
```
ExampleRepo
├── .Makefile.test
│ ├── .....
│ └── Makefile.test
├── Makefile.test -> .Makefile.test/Makefile.test
├── src
│ └── ....
└── test
└── ....
```
### Update your `.gitignore` file.
In order to avoid temporary files that may be created by Makefile.test, you should
to update your `.gitignore` file.
```
# Intermediate files created by Makefile.test
**/.makefile_test_*ed_tests
```
## Killing, Interrupting `make`
If hung tests are encountered, one may want to kill the `make` execution.
For SIGTERM, the user should send SIGTERM to the *process group* of `make`. Using
something similar to:
```
kill -s TERM -- -
```
If SIGTERM is only sent to `make` child processes will be orphaned and left behind.
If `make` is invoked interactively from a terminal, `CTRL-C` should kill all running
processes cleanly.
## Support
Need to contact us directly? Email oss@box.com and be sure to include the name of this project in the subject.
## Copyright and License
Copyright 2017 Box, Inc. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.