Skip to content

Draft: ci/skqp: deqp-runner for skqp

Guilherme Gallo requested to merge gallo/mesa:ci-skqp_deqp-runner into main

On draft due to upcoming changes in skqp-runner binary. Needs testing.

deqp-runner is about to add skqp test suite support. This MR prepares Mesa CI to use it.

Let's use it to easily detect and manage flake tests, which is happening with iris-whl and iris-tgl. This change will also help operate the same format for test expectations and results.

deqp-runner runs each test in one skqp command resulting in distributed result folders. They are located in the results/raw folder. To make it easier for the dev to check the error report, The error_extractor.py plugin aggregates the errors from each command, out.csv, and results.csv are aggregated via a simple cat. So, now we will have just one HTML file for GL, GLES, and VK backends.

Stress tests

Ran the stress test via ci_run_n_monitor script for several hours with no issues.

python .gitlab-ci/bin/ci_run_n_monitor.py --rev $(git rev-parse HEAD) --target '(radeonsi|a630).*skqp.*' --stress
  • radeonsi-raven-skqp:amd64: 45 times (~8 min/job on average)
  • a630_skqp: 27 times (~12 min/job on average)

Next steps (for the next MR)

  • Add support to provide unittests-fails.txt test expectation files
  • Add support to provide rendertests-fails.txt test expectation files
    • Currently, the rendertests generated from list_gms binary are merged with fail caselist.
  • Fix manual jobs iris-whl-skqp and iris-tgl-skqp
  • Update documentation
Edited by Guilherme Gallo

Merge request reports