https://github.com/tom-tan/cwl-conformance-test-summarizer
Summarize the result of conformance test using cwltest
https://github.com/tom-tan/cwl-conformance-test-summarizer
common-workflow-language conformance-tests cwl
Last synced: about 1 year ago
JSON representation
Summarize the result of conformance test using cwltest
- Host: GitHub
- URL: https://github.com/tom-tan/cwl-conformance-test-summarizer
- Owner: tom-tan
- License: mit
- Created: 2018-07-17T07:40:55.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2018-07-18T03:56:06.000Z (almost 8 years ago)
- Last Synced: 2025-02-01T09:13:11.359Z (over 1 year ago)
- Topics: common-workflow-language, conformance-tests, cwl
- Language: Ruby
- Size: 6.84 KB
- Stars: 3
- Watchers: 3
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# cwl-conformance-test-summarizer
This tool summarize the result of [cwltest](https://github.com/common-workflow-language/cwltest) to make the badge using [shields.io](https://shields.io).
Requirements:
- Ruby 2.5.0 or later
Example:
```console
$ pip install cwltest # necessary to run the conformance test
$ pip install cwltool # example of the workflow engine
$ git clone git@github.com:common-workflow-language/common-workflow-language.git
$ cd common-workflow-language
# By default it uses cwltool. For other engines, use ./run_test.sh RUNNER=/path/to/your/favorite/engine
$ ./run_test.sh --junit-xml=results.xml
$ cwl-conformance-test-summarizer.rb v1.0/results.xml | jq .
{
"success": 130,
"success_required": null,
"nall": 132,
"nrequired": 39,
"ratio": 98.5,
"status": "red"
}
```
Once you store the output of this tool in a server that can be accessed via internet,
you can use the badge of the result of conformance test by using the following links, for example:
```
```
Here is the result:
## Todo and Known Issues:
- The badge color should reflect the result. For example, green when all the tests passed, orange when all the tests for required features passed, and red when some tests for required feature failed.
- Currently shields.io does not support dynamic setting for colors. Once [#1525 for badges/shields](https://github.com/badges/shields/pull/1525) is merged, we will use badges that reflect the results.
- I cannot see the badge for the above example...
- Dynamic badge generation in shields.io takes a little time and browsers sometimes fail to get the badge due to timeout.