Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/melezhik/tomty
Tomty - Raku Test Framework
https://github.com/melezhik/tomty
perl6 test-automation test-harness testing
Last synced: 12 days ago
JSON representation
Tomty - Raku Test Framework
- Host: GitHub
- URL: https://github.com/melezhik/tomty
- Owner: melezhik
- Created: 2019-08-07T21:40:46.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2023-11-17T07:57:20.000Z (about 1 year ago)
- Last Synced: 2024-11-05T18:56:48.248Z (about 2 months ago)
- Topics: perl6, test-automation, test-harness, testing
- Language: Raku
- Homepage:
- Size: 112 KB
- Stars: 0
- Watchers: 4
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: Changes
Awesome Lists containing this project
README
# Tomty
Tomty - Raku Test Framework.
# Install
zef install Tomty
# Build Status
[![SparkyCI](https://ci.sparrowhub.io/project/gh-melezhik-Tomty/badge)](https://ci.sparrowhub.io)
# Quick start
tomty --edit test-01
bash "echo Hello World"
tomty --edit test-02
bash "echo Upps && exit 1";
tomty --edit test-03
bash "echo Hello Again!";
tomty --all # Run all tests and make reports
[1/3] / [test-01] ....... 2 sec. OK
[2/3] / [test-02] ....... 3 sec. FAIL
[3/3] / [test-03] ....... 3 sec. OK
=========================================
)=: (2) tests passed / (1) failed# save tests to Git
echo ".tomty/.cache" >> .gitignore
git add .tomty
# Guide
## Writing tests
Tomty test is just a Raku scenario:
tomty --edit test-meta6-file-exist
#!raku
bash "test -f META6.json"
You can write more advanced tests, for example:
# Check if raku.org is accessible
tomty --edit test-raku-org-alive
#!raku
http-ok("https://raku.org");
# Check if META6.json file is a valid json
tomty --edit test-meta6-is-valid-json
#!raku
task-run "meta6 is a valid json", "json-lint";
Check out [Sparrow6 DSL](https://github.com/melezhik/Sparrow6#sparrow6-dsl) on what you can use writing your tests.
## Running tests
* To run all test just say `tomty --all`
It will find all the tests and run them in sequence.
* To run single test just say `tomty $test`
For example:
tomty test-meta6-is-valid-json
## Examining tests
To list all the tests just say `tomty --list`
This command will list all tests.
## Managing tests
### Removing test
To remove test use `--remove` option:
tomty --remove $test
### Edit test source code
Use `--edit` to create test from the scratch or to edit existed test source code:
tomty --edit $test
### Getting test source code
Use `--cat` command to print out test source code:
tomtit --cat $test
Use `--lines` flag to print out test source code with line numbers.
# Environments
* Tomty environments are configuration files, written on Raku and technically speaking are plain Raku Hashes
* Environment configuration files should be placed at `.tomty/conf` directory:
`.tomty/env/config.raku`:
```raku
{
dbname => "products",
dbhost => "localhost"
}
```When tomty runs it picks the `.tomty/env/config.raku` and read configuration from it
variables will be accessible as `config` Hash, inside Tomty scenarios:```raku
my $dbname = config;
my $dbhost = config;
```To define _named_ configuration ( environment ), simply create `.tomty/env/config{$env}.raku` file and refer to it through
`--env=$env` parameter:nano .tomty/env/config.prod.raku
tomty --env=prod ... other parameters here # will run with production configuration
You can run editor for environment configuration by using --edit option:
tomty --env-edit test # edit test environment configuration
tomty --env-edit default # edit default configuration
You can activate environment by using `--env-set` parameter:
tomty --env-set prod # set prod environment as default
tomty --env-set # to list active (current) environment
tomty --env-set default # to set current environment to defaultTo view environment configuration use `--env-cat` command:
tomty --env-cat $env
Use `--lines` flag to print out environment source code with line numbers.
You print out the list of all environments by using `--env-list` parameters:
tomty --env-list
## Macros
Tomty macros allow to pre-process test scenarios. To embed macros use `=begin tomty` .. `=end tomty` syntax:
```raku
=begin tomty
%(
tag => "slow"
)
=end tomty
```Macros could be any Raku code, returning `Hash`. The example above set tag=`slow` for slow running tests,
you can skip test execution by using `--skip` option:tomty --skip=slow
See also `tags filtering`.
Tags could be multiple as well:
```raku
=begin tomty
%(
tag => [ "flaky", "slow" ]
)
=end tomty
```## Tags filtering
Tags filtering allows to run subsets of scenarios using tags as criteria.
### Logical OR
By default logical OR is implied when using comma:
Examples:
tomty --skip=slow,windows # skip slow OR windows tests
tomty --only=frontend,backend # only frontend OR backend test
### Logical AND
Use `+` to mimic logical AND:
tomty --only=database+mysql # execute only mysql database tests
`--skip` and `--only` could be combined to get more sophisticated scenarios:
tomty --only=database+mysql,skip=window # execute only mysql database tests BUT not for windows OS system
## List tags
One can list available tags by:
tomty --list --tags
You can combine `--tags` with `--only` or `--skip` options to _list_ tagged tests.
Examples:
tomty --tags --only=foo # list tests tagged by `foo`
tomty --tags --only=foo+bar # list tests tagged by `foo` AND `bar`
tomty --tags --only=foo,bar # list tests tagged by `foo` OR `bar`
## Filter untagged scenarios
Use special `untagged` keyword to filter untagged scenarios:
```
tomty --tags --only=untagged # list all untagged tests
tomty --only=untagged # run all untagged tests
tomty --skip=untagged # run all but untagged tests
```
## ProfilesTomty profile sets command line arguments for a named profile:
cat .tomty/profile
```raku
%(
default => %(
skip => "broken"
)
)
```One can override following command line arguments through a profile:
* `skip`
* `only`
* `env`
* `no-index-update`In the future more arguments will be supported.
A `default` profile sets default command line arguments when `tomty` cli run.
To add more profiles just add more Hash keys and define proper settings:
```raku
%(
default => %(
skip => "broken"
),
development => %(
only => "new-bugs"
)
)
```
To chose profile use `--profile` option:tomty --profile development
## Bash completion
Tomty comes with nice Bash completion to easy cli usage, use `--completion` option to install completion:
tomty --completion
And then `source ~/.tomty_completion.sh` to activate one.
# Tomty cli
## Options
* `--all|-a`
Run all tests
* `--show-failed`
Show failed tests
* `--verbose`
Runs tests in verbose mode, print more details about every test
* `--no-index-update`
Don't update Sparrow repository index
* `--dump-task`
Dump task code before execution, see SP6_DUMP_TASK_CODE Sparrow documentation
* `--color`
Run in color mode
* `--list`
List tests/tags
To list test only:
tomty --list
To lists tags only:
tomty --list --tags
* `--noheader`
Omit header when list tests. Allow to edit tests one by one:
for i in $(tomty --noheader); do tomty --edit $i; done
* `--edit|--cat|--remove`
Edit, dump, remove test
* `--env-edit|--env-list|--env-set`
Edit, list, set environment
* `--completion`
Install Tomty Bash completion
* `--log`
Get log for given test run, useful when running in all tests mode:
tomty -all
tomty --log test-01
* `--skip` | `--only`
Set tags filters.
Skip tests tagged as `slow`
tomty --skip=slow
Only run tests tagged as `linux`
tomty --only=linux
See also `tags filtering` for more details on tag filtering.
* `--tags`
Show available tags:
tomty --list --tags # list all tags
You can combine `--tags` with `--only` or `--skip` options:
tomty --tags --only=foo+bar # list tests tagged by `foo` AND `bar`
# Environment variables
* `TOMTY_DEBUG`
Use it when debugging Tomty itself:
TOMTY_DEBUG=1 tomty --all
# See also
* [Sparrow6](https://github.com/melezhik/Sparrow6)
* [Tomtit](https://github.com/melezhik/Tomtit)
# Author
Alexey Melezhik
# Thanks to
God, as my inspiration