Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/dodie/testing-in-bash
Bash test framework comparison
https://github.com/dodie/testing-in-bash
bash bash-testing bash-unit bats comparison review shellspec shunit2 testing
Last synced: 7 days ago
JSON representation
Bash test framework comparison
- Host: GitHub
- URL: https://github.com/dodie/testing-in-bash
- Owner: dodie
- License: gpl-2.0
- Created: 2020-04-21T06:52:33.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2024-04-25T07:01:48.000Z (7 months ago)
- Last Synced: 2024-08-01T19:54:16.730Z (3 months ago)
- Topics: bash, bash-testing, bash-unit, bats, comparison, review, shellspec, shunit2, testing
- Language: Shell
- Homepage:
- Size: 93.8 KB
- Stars: 128
- Watchers: 10
- Forks: 10
- Open Issues: 3
-
Metadata Files:
- Readme: readme.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# In search of the best test framework for Bash
There are [many testing frameworks for Bash](https://stackoverflow.com/questions/1339416/unit-testing-bash-scripts).
However, many of them are not actively maintained, or are used only by a small group of people.
In this comparision I've included well-established and new testing frameworks that had at least recent release and have more than 50 Stars on GitHub:
- [bashunit](https://github.com/TypedDevs/bashunit), a fast, simple and flexible testing library for your bash scripts
- [Bats](https://github.com/bats-core/bats-core), a mature testing library with a bit of a custom syntax for more concise tests
- [shUnit2](https://github.com/kward/shunit2), possibly the oldest Bash test framework around, supporting test files in pure Bash
- [bash_unit](https://github.com/pgrange/bash_unit), a new and featureful test framework where test cases can be written in pure Bash
- [ShellSpec](https://github.com/shellspec/shellspec), a BDD style testing framework with its own DSL
- [shpec](https://github.com/rylnd/shpec), a BDD style testing framework similar to RSpec, Jasmine, and mocha
## Test drive
To start fiddling with any of the test projects just run the `install.sh` to download and extract the particular
testing framework to a local directory, then execute `test.sh` to start the suite.
- [bashunit example project and test drive](example-bashunit)
- [Bats example project and test drive](example-bats)
- [shUnit2 example project and test drive](example-shunit2)
- [bash_unit example project and test drive](example-bash_unit)
- [ShellSpec example project and test drive](example-shellspec)
- [shpec example project and test drive](example-shpec)
## Detailed comparision
| | [bashunit](https://github.com/TypedDevs/bashunit) | [Bats](https://github.com/bats-core/bats-core) | [shUnit2](https://github.com/kward/shunit2) | [bash_unit](https://github.com/pgrange/bash_unit) | [ShellSpec](https://github.com/shellspec/shellspec) | [shpec](https://github.com/rylnd/shpec) |
|------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|----------------------------------------------------------------------------|---------------------------------------------------------------------|-----------------------------------------|
| Discover and run all tests
in the project | ✔️ | ✔️ | ❌ [?](example-shunit2#test-discovery) | ⚠ [?](example-bash_unit#test-discovery) | ✔️ [?](example-shellspec#test-discovery) | ✔️ |
| Run subset of tests or
a specific test | ✔️ | ✔️ [?](example-bats#test-discovery) | ⚠ [?](example-shunit2#test-discovery) | ✔️ | ✔️ | ❌ [issue](https://github.com/rylnd/shpec/issues/31) |
| Parallel test execution | ❌ | ✔️ | ❌ | ❌ | ✔️ | ❌ |
| Skip or ignore tests | ✔️ | ✔️ | ⚠ [?](example-shunit2#skip-tests) | ✔️ [?](example-bash_unit#skip-tests) | ✔️ | ❌ [issue](https://github.com/rylnd/shpec/issues/57) |
| Rich assertion library | ✔️ | ⚠ [?](example-bats#assertions) | ⚠ [?](example-shunit2#assertions) | ⚠ [?](example-bash_unit#assertions) | ✔️ [?](example-shellspec#assertions) | ✔️ [?](example-shpec#assertions) |
| Extensible with custom assertions | ✔️ | ✔️ [?](example-bats#custom-assertions) | ✔️ [?](example-shunit2#custom-assertions) | ✔️ [?](example-bash_unit#custom-assertions) | ✔️ [?](example-shellspec#custom-assertions) | ✔️ [?](example-shpec#custom-assertions) |
| Before / After | ✔️ | ✔️ | ⚠ [issue](https://github.com/kward/shunit2/issues/112) | ✔️ | ✔️ | ❌ [issue](https://github.com/rylnd/shpec/issues/52) |
| BeforeAll / AfterAll | ✔️ | ❌ [issue](example-bats#test-discovery) | ⚠ [issue](https://github.com/kward/shunit2/issues/112) | ✔️ | ✔️ [?](example-shellspec#test-format) | ❌ [issue](https://github.com/rylnd/shpec/issues/52) |
| Parameterized tests
and test data | ✔️ | ❌ [issue](https://github.com/sstephenson/bats/issues/136) | ❌ | ❌ | ✔️ | ❌ |
| Mocking \* | ⚠ [issue](https://github.com/TypedDevs/bashunit/pull/232) | ✔️ [?](example-bats#mocking) | ⚠ [?](example-shunit2#mocking) | ✔️ [?](example-bash_unit#mocking) | ✔️ [?](example-shellspec#mocking) | ✔️ [?](example-shpec#mocking) |
| Isolation between tests \*\* | ✔️ | ✔️ [doc](https://github.com/bats-core/bats-core/wiki/Bats-Evaluation-Process) | ❌ | ✔️ | ✔️ | |
| Nice local report | ✔️ | ⚠ [?](example-bats#report) | ✔️ | ✔️ | ✔️ | ✔️ |
| CI compatible report | ⚠ | ✔️ [TAP](http://testanything.org/) | ❌ [issue](https://github.com/kward/shunit2/issues/31) | ✔️ [TAP](http://testanything.org/) | ✔️ [TAP](http://testanything.org/) | ❌ |
| Zero Dependency and
implemented in Bash \*\*\* | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
| Easy to install \*\*\*\* | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
| Tests written in Bash | ✔️ | ⚠ [almost](example-bats#test-format) | ✔️ | ✔️ | ❌ [?](example-shellspec#test-format) | ❌ [?](example-shpec#test-format) |
| Compatible with the
Unofficial Bash Strict Mode \*\*\*\*\* | ⚠ | ✔️| ⚠ [?](https://github.com/dodie/testing-in-bash/tree/master/example-bats#unit-testing-scripts-that-set-bash-options) | ✔️ | ⚠ [issue](https://github.com/pgrange/bash_unit/issues/61) | ✔️ | |
| Maintained | ✔️ | ⚠ [?](example-bats#activity) | ⚠ [?](example-shunit2#activity) | ✔️ [?](example-bash_unit#activity) | ✔️ [?](example-shellspec#activity) | ⚠ [?](example-shpec#activity) |
| Mature | ❌ [since 2023](https://github.com/TypedDevs/bashunit/releases) | ✔️ [since 2011](https://github.com/bats-core/bats-core/releases) | ✔️ [since 2007](https://github.com/kward/shunit2/commits/master?after=0f0a77b6b257f24d2b3ef1b28096fdd154a19f22+10) | ✔️ [since 2016](https://github.com/pgrange/bash_unit/releases) | ❌ [since 2019](https://github.com/shellspec/shellspec/releases) | ✔️ [since 2013](https://github.com/rylnd/shpec/releases) |
| Good documentation | ✔️ [?](https://bashunit.typeddevs.com) | ⚠ [?](example-bats#documentation) | ✔️ [?](example-shunit2#documentation) | ✔️ [?](example-bash_unit#documentation) | ✔️ [?](example-shellspec#documentation) | ✔️ [?](example-shpec#documentation) |
\* **Mocking**:
The testing framework is compatible with the [common mocking techniques](https://github.com/dodie/testing-in-bash/tree/master/mocking).
\*\* **Isolation between tests**:
Aliases, custom function definitions, Bash options and variables are not leaking from one test
to another.
\*\*\* **Zero Dependency and implemented in Bash**:
Apart from a few common tools like `coreutils` and `grep`, you don't need anything to use the framework.
The source code of the framework is itself implemented as Bash scripts. If you are comfortable with Bash, you can read and even modify it.
\*\*\*\* **Easy to install**:
Many of the frameworks support more than one alternatives on how to set it up quickly. It's possible
to install it with your favourite package manager and some of them even have an official Docker image.
Personally, I try to avoid installing project dependencies globally as much as I can,
so I just decided to grab a tar.gz file with the latest release from GitHub.
\*\*\*\*\* **Compatible with the Unofficial Bash Strict Mode**:
Sourcing a script file that use the [Unofficial Bash Strict Mode](http://redsymbol.net/articles/unofficial-bash-strict-mode/) should
not break the test framework. Of course, custom assertions and the test code
[should also work with strict mode](https://github.com/dodie/testing-in-bash/tree/master/strictmode) but that's up for the user.
Also, the test framework shouldn't set Bash options as well to ensure that executing sourced functions work similarly to when they are not sourced.
## Related posts, other comparisions
- https://medium.com/wemake-services/testing-bash-applications-85512e7fe2de
- https://stackoverflow.com/questions/1339416/unit-testing-bash-scripts
- https://www.leadingagile.com/2018/10/unit-testing-shell-scriptspart-one/
## Contribution guide
If you know a testing framework that should be mentioned here or find that there's room for improvement feel free to
open an issue or Pull Request. (See [CONTRIBUTING.md](https://github.com/dodie/testing-in-bash/blob/master/CONTRIBUTING.md).)