https://github.com/exercism/uiua-test-runner
https://github.com/exercism/uiua-test-runner
exercism-test-runner exercism-tooling maintained-autonomous
Last synced: 5 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/exercism/uiua-test-runner
- Owner: exercism
- License: agpl-3.0
- Created: 2024-11-12T08:23:01.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-12-24T12:04:49.000Z (5 months ago)
- Last Synced: 2025-01-07T13:49:18.237Z (5 months ago)
- Topics: exercism-test-runner, exercism-tooling, maintained-autonomous
- Language: Shell
- Size: 39.1 KB
- Stars: 0
- Watchers: 7
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# Exercism Uiua Test Runner
The Docker image to automatically run tests on Uiua solutions submitted to [Exercism].
## Getting started
Build the test runner, conforming to the [Test Runner interface specification](https://github.com/exercism/docs/blob/main/building/tooling/test-runners/interface.md).
Update the files to match your track's needs. At the very least, you'll need to update `bin/run.sh`, `Dockerfile` and the test solutions in the `tests` directory
- Tip: look for `TODO:` comments to point you towards code that need updating
- Tip: look for `OPTIONAL:` comments to point you towards code that _could_ be useful
- Tip: if it proves impossible for the Docker image to work on a read-only filesystem, remove the `--read-only` flag from the `bin/run-in-docker.sh` and `bin/run-tests-in-docker.sh` files.
We don't yet enforce a read-only file system in production, but we might in the future!
## Run the test runner
To run the tests of a single solution, do the following:
1. Open a terminal in the project's root
2. Run `./bin/run.sh `
Once the test runner has finished, its results will be written to `/results.json`.
## Run the test runner on a solution using Docker
_This script is provided for testing purposes, as it mimics how test runners run in Exercism's production environment._
To run the tests of a single solution using the Docker image, do the following:
1. Open a terminal in the project's root
2. Run `./bin/run-in-docker.sh `
Once the test runner has finished, its results will be written to `/results.json`.
## Run the tests
To run the tests to verify the behavior of the test runner, do the following:
1. Open a terminal in the project's root
2. Run `./bin/run-tests.sh`
These are [golden tests][golden] that compare the `results.json` generated by running the current state of the code against the "known good" `tests//expected_results.json`. All files created during the test run itself are discarded.
When you've made modifications to the code that will result in a new "golden" state, you'll need to update the affected `tests//expected_results.json` file(s).
## Run the tests using Docker
_This script is provided for testing purposes, as it mimics how test runners run in Exercism's production environment._
To run the tests to verify the behavior of the test runner using the Docker image, do the following:
1. Open a terminal in the project's root
2. Run `./bin/run-tests-in-docker.sh`
These are [golden tests][golden] that compare the `results.json` generated by running the current state of the code against the "known good" `tests//expected_results.json`. All files created during the test run itself are discarded.
When you've made modifications to the code that will result in a new "golden" state, you'll need to update the affected `tests//expected_results.json` file(s).
## Benchmarking
There are two scripts you can use to benchmark the test runner:
1. `./bin/benchmark.sh`: benchmark the test runner code
2. `./bin/benchmark-in-docker.sh`: benchmark the Docker image
These scripts can give a rough estimation of the test runner's performance.
Bear in mind though that the performance on Exercism's production servers is often lower.
[test-runners]: https://github.com/exercism/docs/tree/main/building/tooling/test-runners
[golden]: https://ro-che.info/articles/2017-12-04-golden-tests
[exercism]: https://exercism.org