https://github.com/flakebi/deqp-runner
A parallel and robust Vulkan Conformance Test Suite runner
https://github.com/flakebi/deqp-runner
cts parallel vulkan
Last synced: 12 months ago
JSON representation
A parallel and robust Vulkan Conformance Test Suite runner
- Host: GitHub
- URL: https://github.com/flakebi/deqp-runner
- Owner: Flakebi
- License: apache-2.0
- Created: 2020-11-22T11:38:16.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2024-04-08T16:46:22.000Z (about 2 years ago)
- Last Synced: 2025-04-06T13:51:50.326Z (about 1 year ago)
- Topics: cts, parallel, vulkan
- Language: Rust
- Homepage:
- Size: 223 KB
- Stars: 1
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE-APACHE
Awesome Lists containing this project
README
# Deqp Runer
Run the Vulkan Conformance Test Suite in parallel and robustly.
This is fork/rewrite of the mesa [parallel deqp runner](https://gitlab.freedesktop.org/mesa/parallel-deqp-runner).
## Features
- Run tests in parallel
- Recover from crashes (e.g. failing asserts)
- Recover from timeouts
- Save results even on fatal errors
- Cross-platform
- Automatically retry failing tests
- Automatically bisect failures if they depend on a combination of tests
- Save a junit compatible xml file of the results
## Usage
Create a `testlist.txt` with one deqp test name per line. Run with
```bash
deqp-runner -t testlist.txt -- ./deqp-vk --deqp-caselist-file
```
A more complicated command:
```bash
deqp-runner -t testlist.txt \
--timeout 10 \
--jobs 2 \
--shuffle \
--no-progress \
--start 50 \
-- ./deqp-vk \
--deqp-log-flush=disable \
--deqp-log-images=disable \
--deqp-log-shader-sources=disable \
--deqp-surface-width=256 \
--deqp-surface-height=256 \
--deqp-surface-type=pbuffer \
--deqp-gl-config-name=rgba8888d24s8ms0 \
--deqp-caselist-file
```
## Output
`deqp-runner` produces a number of output files to collect results and help finding driver issues.
- `summary.csv` contains the result for every test.
- `summary.xml` contains failures and flakes in the junit xml format for consumption by CI systems
like Jenkins.
- `log.json` is written while running. It contains a line for every run test, connecting it with the
PID of `deqp-vk`, and timestamp and a failure directory containing more information. Tests that
are run multiple times because they are flaky are listed multiple times.
- `fails` contains one directory for every `deqp-vk` invocation that had failures. Every failure
subdir contains the list of tests that were run for easy reproducibility and stderr of `deqp-vk`.
`log.json` references failure subdirs for failed tests.
## Build
Install [Rust](https://rust-lang.org) (preferred installation method is [rustup](https://rustup.rs))
and build with `cargo build --release`.
To install without cloning use `cargo install --git https://github.com/Flakebi/deqp-runner`.
## License
Licensed under either of
* [Apache License, Version 2.0](LICENSE-APACHE)
* [MIT license](LICENSE-MIT)
at your option.