Skip to content

ci: New dEQP runner

Emma Anholt requested to merge anholt/mesa:ci-deqp-runner into master
    ci/deqp: Switch to a new dEQP runner written in Rust.
    
    I found the C++ runner hard to develop on, and we had stability issues and
    outstanding feature needs that made me want something I felt good about
    hacking on.  Thus, Rewrite It In Rust of the deqp runner.
    
    The new runner includes:
    
    - Skip lists don't reshuffle the test list.
    - Known-flake handling without resorting to skip lists (fixing our main CI
      reliability issue on a3xx right now).
    - Per-thread Vulkan shader caches should speed up VK CI runtime.
    - Tracking of crashes separate from fails (so we can see progress on that
      front).
    - Logging of deqp stderr spam (particularly assertion failures!) in the CI
      log.
    - Integrated QPA filtering so we don't have bash perf issues for it.
    - Logging of what caselist to go look at for a given error report (in red,
      so it's easier to find in your CI log).
    - The code is 1/3 unit tests, and easy to extend for more coverage.
    - Non-LAVA CI runs create a failures.csv in artifacts that you can check
      in as your deqp-*-fails.txt file.
    - Test runtime is included in results.csv so you can debug how to speed up
      your CI job.
    
    Since this is a new runner with a different RNG, the test groups are
    shuffled one more time.  This seems to result in some panfrost T720
    stability issues (See its new deqp-panfrost-t720-flakes.txt), and one new
    flake in freedreno a630.
Edited by Emma Anholt

Merge request reports