Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/exercism/ballerina-test-runner


https://github.com/exercism/ballerina-test-runner

community-contributions-paused exercism-test-runner exercism-tooling

Last synced: about 1 month ago
JSON representation

Awesome Lists containing this project

README 

        
# Exercism Test Runner Template



This repository is a [template repository](https://help.github.com/en/github/creating-cloning-and-archiving-repositories/creating-a-template-repository) for creating [test runners][test-runners] for [Exercism][exercism] tracks.



## Using the Test Runner Template



1. Ensure that your track has not already implemented a test runner. If there is, there will be a `https://github.com/exercism/-test-runner` repository (i.e. if your track's slug is `python`, the test runner repo would be `https://github.com/exercism/python-test-runner`)

2. Follow [GitHub's documentation](https://help.github.com/en/github/creating-cloning-and-archiving-repositories/creating-a-repository-from-a-template) for creating a repository from a template repository

   - Name your new repository based on your language track's slug (i.e. if your track is for Python, your test runner repo name is `python-test-runner`)

3. Remove this [Exercism Test Runner Template](#exercism-test-runner-template) section from the `README.md` file

4. Build the test runner, conforming to the [Test Runner interface specification](https://github.com/exercism/docs/blob/main/building/tooling/test-runners/interface.md).

   - Update the files to match your track's needs. At the very least, you'll need to update `bin/run.sh`, `Dockerfile` and the test solutions in the `tests` directory

   - Tip: look for `TODO:` comments to point you towards code that need updating

   - Tip: look for `OPTIONAL:` comments to point you towards code that _could_ be useful



Once you're happy with your test runner, [open an issue on the exercism/exercism](https://github.com/exercism/exercism/issues/new?assignees=&labels=&template=new-test-runner.md&title=%5BNew+Test+Runner%5D+) to request an official test runner repository for your track.



# Exercism TRACK_NAME_HERE Test Runner



The Docker image to automatically run tests on TRACK_NAME_HERE solutions submitted to [Exercism].



## Run the test runner



To run the tests of an arbitrary exercise, do the following:



1. Open a terminal in the project's root

2. Run `./bin/run.sh   `



Once the test runner has finished, its results will be written to `/results.json`.



## Run the test runner on an exercise using Docker



_This script is provided for testing purposes, as it mimics how test runners run in Exercism's production environment._



To run the tests of an arbitrary exercise using the Docker image, do the following:



1. Open a terminal in the project's root

2. Run `./bin/run-in-docker.sh   `



Once the test runner has finished, its results will be written to `/results.json`.



## Run the tests



To run the tests to verify the behavior of the test runner, do the following:



1. Open a terminal in the project's root

2. Run `./bin/run-tests.sh`



These are [golden tests][golden] that compare the `results.json` generated by running the current state of the code against the "known good" `tests//results.json`. All files created during the test run itself are discarded.



When you've made modifications to the code that will result in a new "golden" state, you'll need to generate and commit a new `tests//results.json` file.



## Run the tests using Docker



_This script is provided for testing purposes, as it mimics how test runners run in Exercism's production environment._



To run the tests to verify the behavior of the test runner using the Docker image, do the following:



1. Open a terminal in the project's root

2. Run `./bin/run-tests-in-docker.sh`



These are [golden tests][golden] that compare the `results.json` generated by running the current state of the code against the "known good" `tests//results.json`. All files created during the test run itself are discarded.



When you've made modifications to the code that will result in a new "golden" state, you'll need to generate and commit a new `tests//results.json` file.



[test-runners]: https://github.com/exercism/docs/tree/main/building/tooling/test-runners

[golden]: https://ro-che.info/articles/2017-12-04-golden-tests

[exercism]: https://exercism.io