Draft: ci/skqp: deqp-runner for skqp
On draft due to upcoming changes in skqp-runner
binary. Needs testing.
deqp-runner is about to add skqp test suite support. This MR prepares Mesa CI to use it.
Let's use it to easily detect and manage flake tests, which is happening with iris-whl and iris-tgl. This change will also help operate the same format for test expectations and results.
deqp-runner runs each test in one skqp command resulting in distributed
result folders. They are located in the results/raw
folder. To make it
easier for the dev to check the error report, The error_extractor.py
plugin aggregates the errors from each command, out.csv, and results.csv
are aggregated via a simple cat
. So, now we will have just one HTML
file for GL, GLES, and VK backends.
Stress tests
Ran the stress test via ci_run_n_monitor
script for several hours with no issues.
python .gitlab-ci/bin/ci_run_n_monitor.py --rev $(git rev-parse HEAD) --target '(radeonsi|a630).*skqp.*' --stress
- radeonsi-raven-skqp:amd64: 45 times (~8 min/job on average)
- a630_skqp: 27 times (~12 min/job on average)
Next steps (for the next MR)
- Add support to provide
unittests-fails.txt
test expectation files - Add support to provide
rendertests-fails.txt
test expectation files- Currently, the rendertests generated from
list_gms
binary are merged with fail caselist.
- Currently, the rendertests generated from
- Fix manual jobs
iris-whl-skqp
andiris-tgl-skqp
- Update documentation