Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/hernanmd/bisect.sh
Pharo utility to create comparable test results files
https://github.com/hernanmd/bisect.sh
bisection ci pharo pharo-smalltalk sunit
Last synced: 5 days ago
JSON representation
Pharo utility to create comparable test results files
- Host: GitHub
- URL: https://github.com/hernanmd/bisect.sh
- Owner: hernanmd
- License: mit
- Created: 2024-01-24T15:59:24.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2024-01-26T17:29:58.000Z (11 months ago)
- Last Synced: 2024-10-31T05:41:59.038Z (about 2 months ago)
- Topics: bisection, ci, pharo, pharo-smalltalk, sunit
- Language: Shell
- Homepage:
- Size: 9.77 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## Introduction
The script runs the tests in the specified images (with default startup preferences), combines the output into a single XML files, filter test failures, and write two summary text files:
- The tests which failed in "raw" format, meaning as reported by the SUnit reporter.
- The tests which failed in "Class>>selector" format, with one line per failure.
These output files are timestamped so they can be diffed to see when a failure was introduced.
## Sample output
This is a sample output for the "raw" format:
```bash
...
TestFailure
Given Collections do not match!
additions : #('NewTools-SpTextPresenterDecorators')
missing: #()
SystemDependenciesTest(TestAsserter)>>assert:description:resumable:
SystemDependenciesTest(TestAsserter)>>assert:description:
SystemDependenciesTest(TestAsserter)>>assertCollection:hasSameElements:
SystemDependenciesTest>>testExternalIDEDependencies
SystemDependenciesTest(TestCase)>>performTest
TestFailure
Given Collections do not match!
additions : #('NewTools-Morphic')
missing: #()
SystemDependenciesTest(TestAsserter)>>assert:description:resumable:
SystemDependenciesTest(TestAsserter)>>assert:description:
SystemDependenciesTest(TestAsserter)>>assertCollection:hasSameElements:
SystemDependenciesTest>>testExternalSpec2Dependencies
SystemDependenciesTest(TestCase)>>performTest
...
```
This is a sample output for the "Class>>selector" format:
```bash
formatClass: class class ] in RBFormatterTest>>testCoreSystem
RBFormatterTest>>testCoreSystem
ProperMethodCategorizationTest>>testNoUncategorizedMethods
ReleaseTest>>testPharoVersionFileExists
ReleaseTest>>testThatThereAreNoSelectorsRemainingThatAreSentButNotImplemented
SpMorphicBoxLayoutTest>>testBeHomogeneous
SpMorphicBoxLayoutTest>>testBeHomogeneousWorksWhenContractingWindow
SpMorphicBoxLayoutTest>>testBeHomogeneousWorksWhenExpandingWindow
SystemDependenciesTest>>testExternalBasicToolsDependencies
SystemDependenciesTest>>testExternalIDEDependencies
SystemDependenciesTest>>testExternalSpec2Dependencies
SystemDependenciesTest>>testExternalUIDependencies
```
## Usage
The easiest way to execute it is to specify the image file and it will use the VM in the same directory as the image:
```bash
./bisect.sh -i Pharo.image
```
You can also execute the script specifying the full path to image and VM (e.g. for PharoLauncher)
```bash
/bisect.sh -i /Users/mvs/Documents/Pharo/images/Pharo12-SNAPSHOT.build.1243.sha.e4b8f88.arch.64bit/Pharo12-SNAPSHOT.build.1243.sha.e4b8f88.arch.64bit.image -p /Users/mvs/Documents/Pharo/vms/120-x64/Pharo.app/Contents/MacOS/Pharo
```
Prior to each execution, artifact files from previous execution are removed from the directory where the script is ran.
## Known issues
Sometimes the XML output written while executing tests is "malformed", which causes the xpath script to filter the failure nodes to fail. However, I couldn't find so far the cause of this behavior.
This is a sample output during a XPath parse failure:
```bash
not well-formed (invalid token) at line 19658, column 130, byte 2605549:
=================================================================================================================================^
TestFailure
Got File @ /tmp/testSymbolLinkTargetPath105714316634777590822214543053449082639dir/au�( instead of File @ /tmp/testSymbolLinkTargetPath105714316634777590822214543053449082639dir/a.
at /System/Library/Perl/Extras/5.30/darwin-thread-multi-2level/XML/Parser.pm line 187.
```